2024-05-27T08:16:11.845 INFO:root:teuthology version: 0.0.1.dev291+ge7ae758 2024-05-27T08:16:11.845 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --archive /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776 --verbose --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/orig.config.yaml 2024-05-27T08:16:11.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T08:16:11.997 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776 branch: wip-lusov-quiesce description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7727776' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiesce ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: a9cb3a581a309a99b72997ae5ddb88084f5484c9 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: a9cb3a581a309a99b72997ae5ddb88084f5484c9 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiesce sha1: a9cb3a581a309a99b72997ae5ddb88084f5484c9 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5289 sha1: a9cb3a581a309a99b72997ae5ddb88084f5484c9 sleep_before_teardown: 0 subset: 318/5124 suite: fs suite_branch: wip-lusov-quiesce suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a9cb3a581a309a99b72997ae5ddb88084f5484c9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuNm2cXLh+N/H5FHi8UbImUmTUD5/YSdrQjMsgbVCX6wxjRqgImNhPGZtTlJXWaCXiJ04qPsdCICg5XnvUWrrc= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiEv0OlzT2QfJz0mGq/4d6BicUnbNnj4lOlM73rYa4irKB1UhgHyoMkVLDe5TJMhOx+Rbav2msQZC+U8PW/MWw= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD8+razh+UwCY8t3wGtX8tApTDPBugXYdy5SueXxZmeEFM06s9CbztwRmyuh9mvx6iQxqIPd370LDH6sFZ6ogfc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e7ae7580c417d6ab27fda98fecadca5d9f85d63e timestamp: 2024-05-27_07:25:29 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-05-27T08:16:11.997 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa; will attempt to use it 2024-05-27T08:16:11.998 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa/tasks 2024-05-27T08:16:11.998 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-27T08:16:12.000 INFO:teuthology.task.internal:Checking packages... 2024-05-27T08:16:12.022 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a9cb3a581a309a99b72997ae5ddb88084f5484c9' 2024-05-27T08:16:12.022 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-27T08:16:12.022 INFO:teuthology.packaging:ref: None 2024-05-27T08:16:12.022 INFO:teuthology.packaging:tag: None 2024-05-27T08:16:12.022 INFO:teuthology.packaging:branch: wip-lusov-quiesce 2024-05-27T08:16:12.023 INFO:teuthology.packaging:sha1: a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:16:12.023 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiesce 2024-05-27T08:16:12.242 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3868.ga9cb3a58 2024-05-27T08:16:12.244 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-27T08:16:12.250 INFO:teuthology.task.internal:no buildpackages task found 2024-05-27T08:16:12.250 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-27T08:16:12.259 INFO:teuthology.task.internal:Saving configuration 2024-05-27T08:16:12.281 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-27T08:16:12.289 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-27T08:16:12.315 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 08:11:18.903877', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJuNm2cXLh+N/H5FHi8UbImUmTUD5/YSdrQjMsgbVCX6wxjRqgImNhPGZtTlJXWaCXiJ04qPsdCICg5XnvUWrrc='} 2024-05-27T08:16:12.336 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 08:11:18.901928', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIiEv0OlzT2QfJz0mGq/4d6BicUnbNnj4lOlM73rYa4irKB1UhgHyoMkVLDe5TJMhOx+Rbav2msQZC+U8PW/MWw='} 2024-05-27T08:16:12.362 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-27 08:11:18.905594', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD8+razh+UwCY8t3wGtX8tApTDPBugXYdy5SueXxZmeEFM06s9CbztwRmyuh9mvx6iQxqIPd370LDH6sFZ6ogfc='} 2024-05-27T08:16:12.362 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-27T08:16:12.370 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-27T08:16:12.370 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-27T08:16:12.370 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-27T08:16:12.370 INFO:teuthology.run_tasks:Running task console_log... 2024-05-27T08:16:12.469 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f35a7fd3ac0>, signals=[15]) 2024-05-27T08:16:12.469 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-27T08:16:12.477 INFO:teuthology.task.internal:Opening connections... 2024-05-27T08:16:12.477 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:16:12.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T08:16:12.559 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-05-27T08:16:12.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T08:16:12.635 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-05-27T08:16:12.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T08:16:12.707 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-27T08:16:12.713 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-05-27T08:16:12.731 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-05-27T08:16:12.731 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-05-27T08:16:12.788 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T08:16:12.789 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T08:16:12.790 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-05-27T08:16:12.813 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-05-27T08:16:12.830 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-05-27T08:16:12.830 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-05-27T08:16:12.886 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="CentOS Stream" 2024-05-27T08:16:12.886 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="9" 2024-05-27T08:16:12.886 INFO:teuthology.orchestra.run.smithi092.stdout:ID="centos" 2024-05-27T08:16:12.886 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE="rhel fedora" 2024-05-27T08:16:12.886 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="9" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:PLATFORM_ID="platform:el9" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:ANSI_COLOR="0;31" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:LOGO="fedora-logo-icon" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://centos.org/" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T08:16:12.887 INFO:teuthology.orchestra.run.smithi092.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T08:16:12.887 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-05-27T08:16:12.913 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-05-27T08:16:12.931 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-05-27T08:16:12.931 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-05-27T08:16:12.988 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-05-27T08:16:12.988 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-05-27T08:16:12.988 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-05-27T08:16:12.988 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-27T08:16:12.989 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-27T08:16:12.989 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-05-27T08:16:13.013 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-27T08:16:13.023 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-27T08:16:13.031 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-27T08:16:13.031 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-05-27T08:16:13.034 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-05-27T08:16:13.036 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-05-27T08:16:13.052 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-27T08:16:13.058 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-27T08:16:13.059 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-05-27T08:16:13.091 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-05-27T08:16:13.093 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-05-27T08:16:13.196 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-27T08:16:13.273 INFO:teuthology.run_tasks:Running task kernel... 2024-05-27T08:16:13.288 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-05-27T08:16:13.288 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-27T08:16:13.289 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-27T08:16:13.289 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T08:16:13.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:16:13.306 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-05-27T08:16:13.363 INFO:teuthology.orchestra.run.smithi077.stdout:5.14.0-437.el9.x86_64 2024-05-27T08:16:13.363 INFO:teuthology.task.kernel:Running kernel on smithi077: 5.14.0-437.el9.x86_64 2024-05-27T08:16:13.364 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum install -y kernel 2024-05-27T08:16:15.387 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-27T08:16:18.471 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 19 MB 00:02 2024-05-27T08:16:22.796 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.2 MB 00:00 2024-05-27T08:16:25.055 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 17 kB 00:01 2024-05-27T08:16:27.365 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-05-27T08:16:32.548 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-27T08:16:34.472 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-27T08:16:34.472 INFO:teuthology.orchestra.run.smithi077.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-27T08:16:34.562 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-05-27T08:16:34.572 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-05-27T08:16:34.573 INFO:teuthology.orchestra.run.smithi077.stdout: Package Architecture Version Repository Size 2024-05-27T08:16:34.573 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-05-27T08:16:34.573 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-05-27T08:16:34.573 INFO:teuthology.orchestra.run.smithi077.stdout: kernel x86_64 5.14.0-447.el9 baseos 576 k 2024-05-27T08:16:34.573 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-05-27T08:16:34.573 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-core x86_64 5.14.0-447.el9 baseos 16 M 2024-05-27T08:16:34.573 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules x86_64 5.14.0-447.el9 baseos 34 M 2024-05-27T08:16:34.573 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-core x86_64 5.14.0-447.el9 baseos 28 M 2024-05-27T08:16:34.574 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:16:34.574 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-05-27T08:16:34.574 INFO:teuthology.orchestra.run.smithi077.stdout:================================================================================ 2024-05-27T08:16:34.574 INFO:teuthology.orchestra.run.smithi077.stdout:Install 4 Packages 2024-05-27T08:16:34.574 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:16:34.575 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 78 M 2024-05-27T08:16:34.575 INFO:teuthology.orchestra.run.smithi077.stdout:Installed size: 123 M 2024-05-27T08:16:34.575 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-05-27T08:16:35.542 INFO:teuthology.orchestra.run.smithi077.stdout:(1/4): kernel-5.14.0-447.el9.x86_64.rpm 632 kB/s | 576 kB 00:00 2024-05-27T08:16:36.175 INFO:teuthology.orchestra.run.smithi077.stdout:(2/4): kernel-core-5.14.0-447.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-27T08:16:36.892 INFO:teuthology.orchestra.run.smithi077.stdout:(3/4): kernel-modules-5.14.0-447.el9.x86_64.rpm 15 MB/s | 34 MB 00:02 2024-05-27T08:16:37.809 INFO:teuthology.orchestra.run.smithi077.stdout:(4/4): kernel-modules-core-5.14.0-447.el9.x86_6 12 MB/s | 28 MB 00:02 2024-05-27T08:16:37.809 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-05-27T08:16:37.809 INFO:teuthology.orchestra.run.smithi077.stdout:Total 24 MB/s | 78 MB 00:03 2024-05-27T08:16:38.499 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-05-27T08:16:38.840 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-05-27T08:16:38.841 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-05-27T08:16:39.765 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-05-27T08:16:39.766 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-05-27T08:16:42.135 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-05-27T08:16:42.801 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-modules-core-5.14.0-447.el9.x86_64 1/4 2024-05-27T08:16:42.885 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T08:16:45.519 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T08:16:45.650 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T08:16:50.344 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T08:16:50.386 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : kernel-5.14.0-447.el9.x86_64 4/4 2024-05-27T08:16:55.125 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T08:17:24.664 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T08:17:24.702 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-modules-5.14.0-447.el9.x86_64 4/4 2024-05-27T08:17:24.975 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: kernel-5.14.0-447.el9.x86_64 4/4 2024-05-27T08:17:24.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-5.14.0-447.el9.x86_64 1/4 2024-05-27T08:17:24.975 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-core-5.14.0-447.el9.x86_64 2/4 2024-05-27T08:17:24.976 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-modules-5.14.0-447.el9.x86_64 3/4 2024-05-27T08:17:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : kernel-modules-core-5.14.0-447.el9.x86_64 4/4 2024-05-27T08:17:25.783 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:17:25.783 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-05-27T08:17:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-5.14.0-447.el9.x86_64 2024-05-27T08:17:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-core-5.14.0-447.el9.x86_64 2024-05-27T08:17:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-5.14.0-447.el9.x86_64 2024-05-27T08:17:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: kernel-modules-core-5.14.0-447.el9.x86_64 2024-05-27T08:17:25.784 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:17:25.784 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-05-27T08:17:26.083 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-05-27T08:17:26.161 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-27T08:17:26.161 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-27T08:17:26.162 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-447.el9.x86_64 2024-05-27T08:17:26.162 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-27T08:17:26.162 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-27T08:17:26.162 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-27T08:17:26.163 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi077.front.sepia.ceph.com, path=None, version=distro) 2024-05-27T08:17:26.164 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q kernel | sort -rV | head -n 1 2024-05-27T08:17:26.213 INFO:teuthology.orchestra.run.smithi077.stdout:kernel-5.14.0-447.el9.x86_64 2024-05-27T08:17:26.213 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-447.el9.x86_64 2024-05-27T08:17:26.214 DEBUG:teuthology.orchestra.run.smithi077:> sudo rpm -qi grub2-tools 2024-05-27T08:17:26.253 INFO:teuthology.orchestra.run.smithi077.stdout:Name : grub2-tools 2024-05-27T08:17:26.254 INFO:teuthology.orchestra.run.smithi077.stdout:Epoch : 1 2024-05-27T08:17:26.254 INFO:teuthology.orchestra.run.smithi077.stdout:Version : 2.06 2024-05-27T08:17:26.254 INFO:teuthology.orchestra.run.smithi077.stdout:Release : 68.el9 2024-05-27T08:17:26.254 INFO:teuthology.orchestra.run.smithi077.stdout:Architecture: x86_64 2024-05-27T08:17:26.254 INFO:teuthology.orchestra.run.smithi077.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-27T08:17:26.255 INFO:teuthology.orchestra.run.smithi077.stdout:Group : Unspecified 2024-05-27T08:17:26.255 INFO:teuthology.orchestra.run.smithi077.stdout:Size : 8264254 2024-05-27T08:17:26.255 INFO:teuthology.orchestra.run.smithi077.stdout:License : GPLv3+ 2024-05-27T08:17:26.255 INFO:teuthology.orchestra.run.smithi077.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-27T08:17:26.255 INFO:teuthology.orchestra.run.smithi077.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-27T08:17:26.255 INFO:teuthology.orchestra.run.smithi077.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-27T08:17:26.256 INFO:teuthology.orchestra.run.smithi077.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-27T08:17:26.256 INFO:teuthology.orchestra.run.smithi077.stdout:Packager : builder@centos.org 2024-05-27T08:17:26.256 INFO:teuthology.orchestra.run.smithi077.stdout:Vendor : CentOS 2024-05-27T08:17:26.256 INFO:teuthology.orchestra.run.smithi077.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-27T08:17:26.256 INFO:teuthology.orchestra.run.smithi077.stdout:Summary : Support tools for GRUB. 2024-05-27T08:17:26.257 INFO:teuthology.orchestra.run.smithi077.stdout:Description : 2024-05-27T08:17:26.257 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:17:26.257 INFO:teuthology.orchestra.run.smithi077.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-27T08:17:26.257 INFO:teuthology.orchestra.run.smithi077.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-27T08:17:26.257 INFO:teuthology.orchestra.run.smithi077.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-27T08:17:26.257 INFO:teuthology.orchestra.run.smithi077.stdout:hardware devices. 2024-05-27T08:17:26.258 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:17:26.258 INFO:teuthology.orchestra.run.smithi077.stdout:This subpackage provides tools for support of all platforms. 2024-05-27T08:17:26.259 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-27T08:17:26.259 INFO:teuthology.task.kernel:Updating grub on smithi077 to boot 5.14.0-447.el9.x86_64 2024-05-27T08:17:26.260 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-27T08:17:26.867 INFO:teuthology.orchestra.run.smithi077.stderr:Generating grub configuration file ... 2024-05-27T08:17:28.297 INFO:teuthology.orchestra.run.smithi077.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-27T08:17:28.321 INFO:teuthology.orchestra.run.smithi077.stderr:done 2024-05-27T08:17:28.324 DEBUG:teuthology.orchestra.run.smithi077:> mktemp 2024-05-27T08:17:28.339 INFO:teuthology.orchestra.run.smithi077.stdout:/tmp/tmp.UVVI1Gxn78 2024-05-27T08:17:28.339 DEBUG:teuthology.orchestra.run.smithi077:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UVVI1Gxn78 2024-05-27T08:17:28.405 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0666 /tmp/tmp.UVVI1Gxn78 2024-05-27T08:17:28.545 DEBUG:teuthology.orchestra.remote:smithi077:/tmp/tmp.UVVI1Gxn78 is 6KB 2024-05-27T08:17:28.597 DEBUG:teuthology.orchestra.run.smithi077:> rm -fr /tmp/tmp.UVVI1Gxn78 2024-05-27T08:17:28.611 DEBUG:teuthology.orchestra.run.smithi077:> sudo /bin/ls /boot/loader/entries || true 2024-05-27T08:17:28.672 INFO:teuthology.orchestra.run.smithi077.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-27T08:17:28.673 INFO:teuthology.orchestra.run.smithi077.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-27T08:17:28.673 INFO:teuthology.orchestra.run.smithi077.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-27T08:17:28.673 INFO:teuthology.orchestra.run.smithi077.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64.conf 2024-05-27T08:17:28.674 DEBUG:teuthology.orchestra.run.smithi077:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-447.el9.x86_64 2024-05-27T08:17:28.826 DEBUG:teuthology.orchestra.run.smithi077:> sudo shutdown -r now 2024-05-27T08:17:58.829 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-27T08:17:58.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-05-27T08:17:58.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T08:18:58.833 DEBUG:teuthology.orchestra.remote:timed out 2024-05-27T08:19:07.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-05-27T08:19:07.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T08:19:08.259 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-05-27T08:19:08.744 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-05-27T08:19:08.744 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-447.el9.x86_64"... 2024-05-27T08:19:08.745 DEBUG:teuthology.orchestra.run.smithi077:> uname -r 2024-05-27T08:19:08.759 INFO:teuthology.orchestra.run.smithi077.stdout:5.14.0-447.el9.x86_64 2024-05-27T08:19:08.760 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-447.el9.x86_64 vs 5.14.0-447.el9.x86_64 2024-05-27T08:19:08.760 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-27T08:19:08.760 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-27T08:19:09.761 DEBUG:teuthology.parallel:result is None 2024-05-27T08:19:09.762 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-27T08:19:09.769 INFO:teuthology.task.internal:Creating test directory... 2024-05-27T08:19:09.769 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T08:19:09.772 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T08:19:09.774 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-27T08:19:09.792 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-27T08:19:09.798 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-27T08:19:09.804 INFO:teuthology.task.internal:Creating archive directory... 2024-05-27T08:19:09.804 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T08:19:09.828 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T08:19:09.831 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-27T08:19:09.934 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-27T08:19:09.940 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-27T08:19:09.940 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T08:19:09.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:19:09.956 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-27T08:19:09.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:19:09.972 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T08:19:09.975 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T08:19:09.999 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-27T08:19:10.024 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T08:19:10.047 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T08:19:10.063 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T08:19:10.063 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T08:19:10.072 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T08:19:10.132 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-27T08:19:10.134 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-27T08:19:10.140 INFO:teuthology.task.internal:Configuring sudo... 2024-05-27T08:19:10.140 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T08:19:10.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T08:19:10.145 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-27T08:19:10.173 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-27T08:19:10.182 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-27T08:19:10.182 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T08:19:10.209 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T08:19:10.211 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-27T08:19:10.231 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T08:19:10.305 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T08:19:10.351 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T08:19:10.436 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T08:19:10.474 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:19:10.474 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T08:19:10.554 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T08:19:10.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T08:19:10.644 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T08:19:10.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T08:19:10.772 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:19:10.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T08:19:10.840 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T08:19:10.887 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-27T08:19:10.939 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T08:19:11.024 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-27T08:19:11.067 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:19:11.067 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-27T08:19:11.134 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-05-27T08:19:11.136 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-05-27T08:19:11.138 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-05-27T08:19:11.168 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:19:11.175 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:19:11.212 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:19:11.649 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-27T08:19:11.659 INFO:teuthology.task.internal:Starting timer... 2024-05-27T08:19:11.659 INFO:teuthology.run_tasks:Running task pcp... 2024-05-27T08:19:11.672 INFO:teuthology.run_tasks:Running task selinux... 2024-05-27T08:19:11.685 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-27T08:19:11.685 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-05-27T08:19:11.767 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-05-27T08:19:11.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo service auditd rotate 2024-05-27T08:19:11.846 INFO:teuthology.orchestra.run.smithi092.stdout:Rotating logs: 2024-05-27T08:19:11.848 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-05-27T08:19:11.925 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-05-27T08:19:11.927 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-27T08:19:11.927 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-05-27T08:19:11.959 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-05-27T08:19:11.959 DEBUG:teuthology.orchestra.run.smithi092:> /usr/sbin/getenforce 2024-05-27T08:19:11.980 INFO:teuthology.orchestra.run.smithi092.stdout:Permissive 2024-05-27T08:19:11.980 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-05-27T08:19:12.038 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-05-27T08:19:12.039 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi092.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-05-27T08:19:12.039 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:19:12.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:19:12.065 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:19:12.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:19:12.095 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:19:12.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:19:12.123 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-27T08:19:12.124 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-05-27T08:19:12.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo /usr/sbin/setenforce permissive 2024-05-27T08:19:12.183 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-05-27T08:19:12.216 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-27T08:19:12.225 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-27T08:19:12.228 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-27T08:19:12.355 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-27T08:19:12.365 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-27T08:19:12.366 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi077.front.sepia.ceph.com,smithi092.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-27T08:24:02.561 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-05-27T08:24:02.563 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-05-27T08:24:02.564 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T08:24:02.646 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-05-27T08:24:02.721 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-05-27T08:24:02.722 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-27T08:24:02.722 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T08:24:02.799 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-05-27T08:24:02.873 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-05-27T08:24:02.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-27T08:24:02.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-27T08:24:02.947 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-05-27T08:24:03.039 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-05-27T08:24:03.039 INFO:teuthology.run_tasks:Running task clock... 2024-05-27T08:24:03.050 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-27T08:24:03.051 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T08:24:03.051 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:24:03.054 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T08:24:03.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:24:03.056 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-27T08:24:03.056 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:24:03.081 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T08:24:03.083 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T08:24:03.095 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T08:24:03.097 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T08:24:03.111 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-27T08:24:03.118 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-05-27T08:24:03.120 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: ntpd: command not found 2024-05-27T08:24:03.125 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-27T08:24:03.129 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-05-27T08:24:03.131 INFO:teuthology.orchestra.run.smithi092.stdout:506 Cannot talk to daemon 2024-05-27T08:24:03.143 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T08:24:03.144 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T08:24:03.150 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-05-27T08:24:03.157 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T08:24:03.157 INFO:teuthology.orchestra.run.smithi092.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T08:24:03.160 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-05-27T08:24:03.174 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-27T08:24:03.187 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-27T08:24:03.212 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:24:03.214 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:24:03.216 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:24:03.216 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-05-27T08:24:03.216 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.216 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.216 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.216 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.218 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:24:03.218 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-27T08:24:03.218 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.218 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.218 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.218 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.245 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:24:03.248 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:24:03.248 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-05-27T08:24:03.249 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.249 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.249 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.249 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:24:03.250 INFO:teuthology.run_tasks:Running task install... 2024-05-27T08:24:03.258 DEBUG:teuthology.task.install:project ceph 2024-05-27T08:24:03.258 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a9cb3a581a309a99b72997ae5ddb88084f5484c9'}} 2024-05-27T08:24:03.258 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a9cb3a581a309a99b72997ae5ddb88084f5484c9'} 2024-05-27T08:24:03.258 INFO:teuthology.task.install:Using flavor: default 2024-05-27T08:24:03.264 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-27T08:24:03.264 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-27T08:24:03.264 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a9cb3a581a309a99b72997ae5ddb88084f5484c9', 'tag': None, 'wait_for_package': False} 2024-05-27T08:24:03.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:24:03.267 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a9cb3a581a309a99b72997ae5ddb88084f5484c9', 'tag': None, 'wait_for_package': False} 2024-05-27T08:24:03.267 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:24:03.268 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a9cb3a581a309a99b72997ae5ddb88084f5484c9', 'tag': None, 'wait_for_package': False} 2024-05-27T08:24:03.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:24:03.417 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/ 2024-05-27T08:24:03.417 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3868.ga9cb3a58 2024-05-27T08:24:03.429 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/ 2024-05-27T08:24:03.429 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3868.ga9cb3a58 2024-05-27T08:24:03.438 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/ 2024-05-27T08:24:03.438 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3868.ga9cb3a58 2024-05-27T08:24:03.595 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T08:24:03.595 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:24:03.595 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T08:24:03.607 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T08:24:03.608 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:24:03.608 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T08:24:03.621 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-27T08:24:03.622 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:24:03.622 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-27T08:24:03.623 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T08:24:03.624 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a9cb3a581a309a99b72997ae5ddb88084f5484c9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T08:24:03.632 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T08:24:03.632 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a9cb3a581a309a99b72997ae5ddb88084f5484c9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T08:24:03.648 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-27T08:24:03.648 DEBUG:teuthology.orchestra.run.smithi092:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a9cb3a581a309a99b72997ae5ddb88084f5484c9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-27T08:24:03.690 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T08:24:03.697 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T08:24:03.714 DEBUG:teuthology.orchestra.run.smithi092:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-27T08:24:03.765 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T08:24:03.774 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T08:24:03.790 DEBUG:teuthology.orchestra.run.smithi092:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-27T08:24:03.834 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-05-27T08:24:03.836 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-05-27T08:24:03.839 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-05-27T08:24:03.840 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-05-27T08:24:03.856 INFO:teuthology.orchestra.run.smithi092.stdout:check_obsoletes = 1 2024-05-27T08:24:03.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum clean all 2024-05-27T08:24:04.059 INFO:teuthology.orchestra.run.smithi077.stdout:45 files removed 2024-05-27T08:24:04.068 INFO:teuthology.orchestra.run.smithi179.stdout:45 files removed 2024-05-27T08:24:04.085 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T08:24:04.086 INFO:teuthology.orchestra.run.smithi092.stdout:45 files removed 2024-05-27T08:24:04.095 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T08:24:04.112 DEBUG:teuthology.orchestra.run.smithi092:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-27T08:24:04.759 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 209 kB/s | 83 kB 00:00 2024-05-27T08:24:04.766 INFO:teuthology.orchestra.run.smithi092.stdout:ceph packages for x86_64 211 kB/s | 83 kB 00:00 2024-05-27T08:24:04.769 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-05-27T08:24:05.090 INFO:teuthology.orchestra.run.smithi092.stdout:ceph noarch packages 43 kB/s | 12 kB 00:00 2024-05-27T08:24:05.118 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-05-27T08:24:05.170 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-05-27T08:24:05.450 INFO:teuthology.orchestra.run.smithi092.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-05-27T08:24:05.477 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-05-27T08:24:05.521 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-05-27T08:24:06.063 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-27T08:24:06.075 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-05-27T08:24:07.335 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.1 MB 00:01 2024-05-27T08:24:07.692 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - AppStream 35 MB/s | 19 MB 00:00 2024-05-27T08:24:07.940 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-27T08:24:11.595 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-05-27T08:24:11.843 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-27T08:24:12.016 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-27T08:24:13.309 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 115 kB/s | 17 kB 00:00 2024-05-27T08:24:13.462 INFO:teuthology.orchestra.run.smithi092.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 17 kB 00:00 2024-05-27T08:24:15.604 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-05-27T08:24:15.803 INFO:teuthology.orchestra.run.smithi092.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 21 MB 00:02 2024-05-27T08:24:16.587 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-05-27T08:24:18.421 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 17 kB 00:00 2024-05-27T08:24:19.893 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-27T08:24:20.735 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-05-27T08:24:20.995 INFO:teuthology.orchestra.run.smithi092.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-27T08:24:22.904 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T08:24:22.904 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T08:24:22.915 INFO:teuthology.orchestra.run.smithi179.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T08:24:22.915 INFO:teuthology.orchestra.run.smithi179.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T08:24:22.916 INFO:teuthology.orchestra.run.smithi179.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T08:24:22.917 INFO:teuthology.orchestra.run.smithi179.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T08:24:22.919 INFO:teuthology.orchestra.run.smithi179.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T08:24:22.921 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T08:24:22.922 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T08:24:22.983 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-05-27T08:24:23.006 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-05-27T08:24:23.006 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-05-27T08:24:23.006 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-05-27T08:24:23.006 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-05-27T08:24:23.006 INFO:teuthology.orchestra.run.smithi179.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 6.6 k 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 5.4 M 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 890 k 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 145 k 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 1.5 M 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 143 k 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 3.5 M 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 7.4 M 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 49 k 2024-05-27T08:24:23.007 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 11 M 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 49 M 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 273 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 763 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 225 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 27 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 18 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 32 k 2024-05-27T08:24:23.008 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 747 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 127 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 161 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 321 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 300 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 100 k 2024-05-27T08:24:23.009 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 85 k 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.1 M 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 171 k 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T08:24:23.010 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.4 M 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.1 M 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 21 M 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 34 k 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 2.4 M 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 274 k 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 4.7 M 2024-05-27T08:24:23.011 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 17 M 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 16 k 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 25 k 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T08:24:23.012 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 164 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 501 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 5.4 M 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T08:24:23.013 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T08:24:23.014 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 45 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 136 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T08:24:23.015 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T08:24:23.016 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T08:24:23.017 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T08:24:23.018 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T08:24:23.019 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout:Install 148 Packages 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 11 Packages 2024-05-27T08:24:23.020 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:24:23.021 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 227 M 2024-05-27T08:24:23.021 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-05-27T08:24:23.129 INFO:teuthology.orchestra.run.smithi092.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T08:24:23.129 INFO:teuthology.orchestra.run.smithi092.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T08:24:23.140 INFO:teuthology.orchestra.run.smithi092.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T08:24:23.141 INFO:teuthology.orchestra.run.smithi092.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T08:24:23.142 INFO:teuthology.orchestra.run.smithi092.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T08:24:23.143 INFO:teuthology.orchestra.run.smithi092.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T08:24:23.144 INFO:teuthology.orchestra.run.smithi092.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T08:24:23.147 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T08:24:23.147 INFO:teuthology.orchestra.run.smithi092.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T08:24:23.209 INFO:teuthology.orchestra.run.smithi092.stdout:Dependencies resolved. 2024-05-27T08:24:23.228 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-27T08:24:23.228 INFO:teuthology.orchestra.run.smithi092.stdout: Package Arch Version Repository Size 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout:Installing: 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 6.6 k 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 5.4 M 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 890 k 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 145 k 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 1.5 M 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 143 k 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 3.5 M 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 7.4 M 2024-05-27T08:24:23.229 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 49 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 11 M 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 49 M 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 273 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 763 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 225 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 27 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 18 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 32 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 747 k 2024-05-27T08:24:23.230 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 127 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 161 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 321 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 300 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 100 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 85 k 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.1 M 2024-05-27T08:24:23.231 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 171 k 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrading: 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T08:24:23.232 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.4 M 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.1 M 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout:Installing dependencies: 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 21 M 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 34 k 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 2.4 M 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 274 k 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 4.7 M 2024-05-27T08:24:23.233 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 17 M 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 16 k 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 25 k 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T08:24:23.234 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 164 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 501 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 5.4 M 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T08:24:23.235 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T08:24:23.236 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 45 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 136 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T08:24:23.237 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T08:24:23.238 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T08:24:23.239 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T08:24:23.240 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T08:24:23.241 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T08:24:23.242 INFO:teuthology.orchestra.run.smithi092.stdout:Installing weak dependencies: 2024-05-27T08:24:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T08:24:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:24:23.242 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction Summary 2024-05-27T08:24:23.242 INFO:teuthology.orchestra.run.smithi092.stdout:======================================================================================= 2024-05-27T08:24:23.242 INFO:teuthology.orchestra.run.smithi092.stdout:Install 148 Packages 2024-05-27T08:24:23.242 INFO:teuthology.orchestra.run.smithi092.stdout:Upgrade 11 Packages 2024-05-27T08:24:23.242 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:24:23.243 INFO:teuthology.orchestra.run.smithi092.stdout:Total download size: 227 M 2024-05-27T08:24:23.243 INFO:teuthology.orchestra.run.smithi092.stdout:Downloading Packages: 2024-05-27T08:24:23.837 INFO:teuthology.orchestra.run.smithi179.stdout:(1/159): ceph-19.0.0-3868.ga9cb3a58.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-05-27T08:24:24.345 INFO:teuthology.orchestra.run.smithi092.stdout:(1/159): ceph-19.0.0-3868.ga9cb3a58.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-05-27T08:24:24.396 INFO:teuthology.orchestra.run.smithi179.stdout:(2/159): ceph-fuse-19.0.0-3868.ga9cb3a58.el9.x8 1.6 MB/s | 890 kB 00:00 2024-05-27T08:24:24.580 INFO:teuthology.orchestra.run.smithi179.stdout:(3/159): ceph-immutable-object-cache-19.0.0-386 789 kB/s | 145 kB 00:00 2024-05-27T08:24:24.997 INFO:teuthology.orchestra.run.smithi179.stdout:(4/159): ceph-common-19.0.0-3868.ga9cb3a58.el9. 15 MB/s | 21 MB 00:01 2024-05-27T08:24:25.147 INFO:teuthology.orchestra.run.smithi179.stdout:(5/159): ceph-mgr-19.0.0-3868.ga9cb3a58.el9.x86 10 MB/s | 1.5 MB 00:00 2024-05-27T08:24:25.179 INFO:teuthology.orchestra.run.smithi092.stdout:(2/159): ceph-fuse-19.0.0-3868.ga9cb3a58.el9.x8 1.0 MB/s | 890 kB 00:00 2024-05-27T08:24:25.214 INFO:teuthology.orchestra.run.smithi179.stdout:(6/159): ceph-base-19.0.0-3868.ga9cb3a58.el9.x8 3.4 MB/s | 5.4 MB 00:01 2024-05-27T08:24:25.249 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 5.5 kB/s | 1.7 kB 00:00 2024-05-27T08:24:25.338 INFO:teuthology.orchestra.run.smithi092.stdout:(3/159): ceph-immutable-object-cache-19.0.0-386 914 kB/s | 145 kB 00:00 2024-05-27T08:24:25.481 INFO:teuthology.orchestra.run.smithi179.stdout:(7/159): ceph-mon-19.0.0-3868.ga9cb3a58.el9.x86 14 MB/s | 4.7 MB 00:00 2024-05-27T08:24:25.564 INFO:teuthology.orchestra.run.smithi179.stdout:(8/159): ceph-mds-19.0.0-3868.ga9cb3a58.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-05-27T08:24:25.648 INFO:teuthology.orchestra.run.smithi179.stdout:(9/159): ceph-selinux-19.0.0-3868.ga9cb3a58.el9 302 kB/s | 25 kB 00:00 2024-05-27T08:24:26.338 INFO:teuthology.orchestra.run.smithi092.stdout:(4/159): ceph-mds-19.0.0-3868.ga9cb3a58.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-05-27T08:24:26.463 INFO:teuthology.orchestra.run.smithi092.stdout:(5/159): ceph-base-19.0.0-3868.ga9cb3a58.el9.x8 2.2 MB/s | 5.4 MB 00:02 2024-05-27T08:24:26.938 INFO:teuthology.orchestra.run.smithi092.stdout:(6/159): ceph-mgr-19.0.0-3868.ga9cb3a58.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-05-27T08:24:27.057 INFO:teuthology.orchestra.run.smithi179.stdout:(10/159): ceph-radosgw-19.0.0-3868.ga9cb3a58.el 6.7 MB/s | 11 MB 00:01 2024-05-27T08:24:27.140 INFO:teuthology.orchestra.run.smithi179.stdout:(11/159): cephfs-mirror-19.0.0-3868.ga9cb3a58.e 2.6 MB/s | 225 kB 00:00 2024-05-27T08:24:27.207 INFO:teuthology.orchestra.run.smithi179.stdout:(12/159): cephfs-shell-19.0.0-3868.ga9cb3a58.el 405 kB/s | 27 kB 00:00 2024-05-27T08:24:27.274 INFO:teuthology.orchestra.run.smithi179.stdout:(13/159): libcephfs-devel-19.0.0-3868.ga9cb3a58 476 kB/s | 32 kB 00:00 2024-05-27T08:24:27.441 INFO:teuthology.orchestra.run.smithi179.stdout:(14/159): libcephfs2-19.0.0-3868.ga9cb3a58.el9. 4.4 MB/s | 747 kB 00:00 2024-05-27T08:24:27.458 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T08:24:27.459 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-27T08:24:27.470 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-27T08:24:27.471 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-27T08:24:27.472 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-27T08:24:27.473 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-27T08:24:27.474 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-27T08:24:27.476 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-27T08:24:27.477 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-27T08:24:27.525 INFO:teuthology.orchestra.run.smithi179.stdout:(15/159): libcephsqlite-19.0.0-3868.ga9cb3a58.e 1.9 MB/s | 164 kB 00:00 2024-05-27T08:24:27.540 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-05-27T08:24:27.562 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 6.6 k 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 5.4 M 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 890 k 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 145 k 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 1.5 M 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 143 k 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 3.5 M 2024-05-27T08:24:27.563 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 7.4 M 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 49 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 11 M 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 49 M 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 273 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 763 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 225 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 27 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 18 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 32 k 2024-05-27T08:24:27.564 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 747 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 127 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 161 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 321 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 300 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 100 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 85 k 2024-05-27T08:24:27.565 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.1 M 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 171 k 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager x86_64 1:1.47.90-1.el9 baseos 2.3 M 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm x86_64 1:1.47.90-1.el9 baseos 1.8 M 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team x86_64 1:1.47.90-1.el9 baseos 41 k 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui x86_64 1:1.47.90-1.el9 baseos 248 k 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.4 M 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 3.1 M 2024-05-27T08:24:27.566 INFO:teuthology.orchestra.run.smithi077.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 21 M 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 34 k 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 2.4 M 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 274 k 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 4.7 M 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 17 M 2024-05-27T08:24:27.567 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3868.ga9cb3a58.el9 ceph-noarch 16 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 25 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 164 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-27T08:24:27.568 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 501 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 5.4 M 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-27T08:24:27.569 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 45 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:19.0.0-3868.ga9cb3a58.el9 ceph 136 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-27T08:24:27.570 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-27T08:24:27.571 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-27T08:24:27.572 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-27T08:24:27.573 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-27T08:24:27.574 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:24:27.575 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-05-27T08:24:27.576 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-05-27T08:24:27.576 INFO:teuthology.orchestra.run.smithi077.stdout:Install 148 Packages 2024-05-27T08:24:27.576 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 11 Packages 2024-05-27T08:24:27.576 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:24:27.576 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 227 M 2024-05-27T08:24:27.576 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-05-27T08:24:27.651 INFO:teuthology.orchestra.run.smithi179.stdout:(16/159): librados-devel-19.0.0-3868.ga9cb3a58. 1.0 MB/s | 127 kB 00:00 2024-05-27T08:24:27.767 INFO:teuthology.orchestra.run.smithi179.stdout:(17/159): libradosstriper1-19.0.0-3868.ga9cb3a5 4.2 MB/s | 501 kB 00:00 2024-05-27T08:24:27.847 INFO:teuthology.orchestra.run.smithi092.stdout:(7/159): ceph-common-19.0.0-3868.ga9cb3a58.el9. 5.5 MB/s | 21 MB 00:03 2024-05-27T08:24:28.300 INFO:teuthology.orchestra.run.smithi077.stdout:(1/159): ceph-19.0.0-3868.ga9cb3a58.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-05-27T08:24:28.355 INFO:teuthology.orchestra.run.smithi092.stdout:(8/159): ceph-mon-19.0.0-3868.ga9cb3a58.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-05-27T08:24:28.447 INFO:teuthology.orchestra.run.smithi092.stdout:(9/159): ceph-selinux-19.0.0-3868.ga9cb3a58.el9 275 kB/s | 25 kB 00:00 2024-05-27T08:24:28.642 INFO:teuthology.orchestra.run.smithi179.stdout:(18/159): librgw2-19.0.0-3868.ga9cb3a58.el9.x86 6.1 MB/s | 5.4 MB 00:00 2024-05-27T08:24:28.726 INFO:teuthology.orchestra.run.smithi179.stdout:(19/159): python3-ceph-argparse-19.0.0-3868.ga9 542 kB/s | 45 kB 00:00 2024-05-27T08:24:28.826 INFO:teuthology.orchestra.run.smithi179.stdout:(20/159): python3-ceph-common-19.0.0-3868.ga9cb 1.3 MB/s | 136 kB 00:00 2024-05-27T08:24:28.927 INFO:teuthology.orchestra.run.smithi179.stdout:(21/159): python3-cephfs-19.0.0-3868.ga9cb3a58. 1.6 MB/s | 161 kB 00:00 2024-05-27T08:24:29.035 INFO:teuthology.orchestra.run.smithi179.stdout:(22/159): python3-rados-19.0.0-3868.ga9cb3a58.e 2.9 MB/s | 321 kB 00:00 2024-05-27T08:24:29.136 INFO:teuthology.orchestra.run.smithi179.stdout:(23/159): python3-rbd-19.0.0-3868.ga9cb3a58.el9 2.9 MB/s | 300 kB 00:00 2024-05-27T08:24:29.220 INFO:teuthology.orchestra.run.smithi179.stdout:(24/159): python3-rgw-19.0.0-3868.ga9cb3a58.el9 1.2 MB/s | 100 kB 00:00 2024-05-27T08:24:29.258 INFO:teuthology.orchestra.run.smithi077.stdout:(2/159): ceph-fuse-19.0.0-3868.ga9cb3a58.el9.x8 929 kB/s | 890 kB 00:00 2024-05-27T08:24:29.303 INFO:teuthology.orchestra.run.smithi179.stdout:(25/159): rbd-fuse-19.0.0-3868.ga9cb3a58.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-27T08:24:29.425 INFO:teuthology.orchestra.run.smithi077.stdout:(3/159): ceph-immutable-object-cache-19.0.0-386 869 kB/s | 145 kB 00:00 2024-05-27T08:24:29.487 INFO:teuthology.orchestra.run.smithi179.stdout:(26/159): ceph-osd-19.0.0-3868.ga9cb3a58.el9.x8 3.9 MB/s | 17 MB 00:04 2024-05-27T08:24:29.604 INFO:teuthology.orchestra.run.smithi179.stdout:(27/159): rbd-nbd-19.0.0-3868.ga9cb3a58.el9.x86 1.4 MB/s | 171 kB 00:00 2024-05-27T08:24:29.713 INFO:teuthology.orchestra.run.smithi179.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3868.g 316 kB/s | 34 kB 00:00 2024-05-27T08:24:29.792 INFO:teuthology.orchestra.run.smithi077.stdout:(4/159): ceph-base-19.0.0-3868.ga9cb3a58.el9.x8 2.8 MB/s | 5.4 MB 00:01 2024-05-27T08:24:29.822 INFO:teuthology.orchestra.run.smithi179.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3868.ga9cb3a5 1.3 MB/s | 143 kB 00:00 2024-05-27T08:24:29.880 INFO:teuthology.orchestra.run.smithi179.stdout:(30/159): rbd-mirror-19.0.0-3868.ga9cb3a58.el9. 5.4 MB/s | 3.1 MB 00:00 2024-05-27T08:24:30.097 INFO:teuthology.orchestra.run.smithi092.stdout:(10/159): ceph-radosgw-19.0.0-3868.ga9cb3a58.el 4.7 MB/s | 11 MB 00:02 2024-05-27T08:24:30.176 INFO:teuthology.orchestra.run.smithi077.stdout:(5/159): ceph-mgr-19.0.0-3868.ga9cb3a58.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-05-27T08:24:30.272 INFO:teuthology.orchestra.run.smithi092.stdout:(11/159): ceph-osd-19.0.0-3868.ga9cb3a58.el9.x8 5.1 MB/s | 17 MB 00:03 2024-05-27T08:24:30.297 INFO:teuthology.orchestra.run.smithi092.stdout:(12/159): cephfs-mirror-19.0.0-3868.ga9cb3a58.e 1.1 MB/s | 225 kB 00:00 2024-05-27T08:24:30.373 INFO:teuthology.orchestra.run.smithi092.stdout:(13/159): cephfs-shell-19.0.0-3868.ga9cb3a58.el 271 kB/s | 27 kB 00:00 2024-05-27T08:24:30.398 INFO:teuthology.orchestra.run.smithi092.stdout:(14/159): libcephfs-devel-19.0.0-3868.ga9cb3a58 316 kB/s | 32 kB 00:00 2024-05-27T08:24:30.468 INFO:teuthology.orchestra.run.smithi077.stdout:(6/159): ceph-mds-19.0.0-3868.ga9cb3a58.el9.x86 2.3 MB/s | 2.4 MB 00:01 2024-05-27T08:24:30.498 INFO:teuthology.orchestra.run.smithi092.stdout:(15/159): libcephsqlite-19.0.0-3868.ga9cb3a58.e 1.6 MB/s | 164 kB 00:00 2024-05-27T08:24:30.599 INFO:teuthology.orchestra.run.smithi092.stdout:(16/159): librados-devel-19.0.0-3868.ga9cb3a58. 1.2 MB/s | 127 kB 00:00 2024-05-27T08:24:30.632 INFO:teuthology.orchestra.run.smithi092.stdout:(17/159): libcephfs2-19.0.0-3868.ga9cb3a58.el9. 2.8 MB/s | 747 kB 00:00 2024-05-27T08:24:30.972 INFO:teuthology.orchestra.run.smithi179.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-05-27T08:24:31.048 INFO:teuthology.orchestra.run.smithi179.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3868.ga9 3.6 MB/s | 274 kB 00:00 2024-05-27T08:24:31.074 INFO:teuthology.orchestra.run.smithi092.stdout:(18/159): libradosstriper1-19.0.0-3868.ga9cb3a5 1.0 MB/s | 501 kB 00:00 2024-05-27T08:24:31.106 INFO:teuthology.orchestra.run.smithi179.stdout:(33/159): ceph-mgr-dashboard-19.0.0-3868.ga9cb3 2.8 MB/s | 3.5 MB 00:01 2024-05-27T08:24:31.123 INFO:teuthology.orchestra.run.smithi179.stdout:(34/159): ceph-mgr-rook-19.0.0-3868.ga9cb3a58.e 656 kB/s | 49 kB 00:00 2024-05-27T08:24:31.182 INFO:teuthology.orchestra.run.smithi179.stdout:(35/159): ceph-volume-19.0.0-3868.ga9cb3a58.el9 4.6 MB/s | 273 kB 00:00 2024-05-27T08:24:31.199 INFO:teuthology.orchestra.run.smithi179.stdout:(36/159): ceph-prometheus-alerts-19.0.0-3868.ga 178 kB/s | 16 kB 00:00 2024-05-27T08:24:31.233 INFO:teuthology.orchestra.run.smithi092.stdout:(19/159): python3-ceph-argparse-19.0.0-3868.ga9 285 kB/s | 45 kB 00:00 2024-05-27T08:24:31.251 INFO:teuthology.orchestra.run.smithi077.stdout:(7/159): ceph-mon-19.0.0-3868.ga9cb3a58.el9.x86 4.4 MB/s | 4.7 MB 00:01 2024-05-27T08:24:31.266 INFO:teuthology.orchestra.run.smithi179.stdout:(37/159): cephfs-top-19.0.0-3868.ga9cb3a58.el9. 273 kB/s | 18 kB 00:00 2024-05-27T08:24:31.300 INFO:teuthology.orchestra.run.smithi092.stdout:(20/159): python3-ceph-common-19.0.0-3868.ga9cb 2.0 MB/s | 136 kB 00:00 2024-05-27T08:24:31.308 INFO:teuthology.orchestra.run.smithi179.stdout:(38/159): cephadm-19.0.0-3868.ga9cb3a58.el9.noa 5.9 MB/s | 763 kB 00:00 2024-05-27T08:24:31.367 INFO:teuthology.orchestra.run.smithi092.stdout:(21/159): python3-cephfs-19.0.0-3868.ga9cb3a58. 2.4 MB/s | 161 kB 00:00 2024-05-27T08:24:31.483 INFO:teuthology.orchestra.run.smithi092.stdout:(22/159): python3-rados-19.0.0-3868.ga9cb3a58.e 2.7 MB/s | 321 kB 00:00 2024-05-27T08:24:31.642 INFO:teuthology.orchestra.run.smithi092.stdout:(23/159): librgw2-19.0.0-3868.ga9cb3a58.el9.x86 5.3 MB/s | 5.4 MB 00:01 2024-05-27T08:24:31.667 INFO:teuthology.orchestra.run.smithi092.stdout:(24/159): python3-rbd-19.0.0-3868.ga9cb3a58.el9 1.6 MB/s | 300 kB 00:00 2024-05-27T08:24:31.709 INFO:teuthology.orchestra.run.smithi092.stdout:(25/159): python3-rgw-19.0.0-3868.ga9cb3a58.el9 1.5 MB/s | 100 kB 00:00 2024-05-27T08:24:31.735 INFO:teuthology.orchestra.run.smithi092.stdout:(26/159): rbd-fuse-19.0.0-3868.ga9cb3a58.el9.x8 1.2 MB/s | 85 kB 00:00 2024-05-27T08:24:31.810 INFO:teuthology.orchestra.run.smithi092.stdout:(27/159): rbd-nbd-19.0.0-3868.ga9cb3a58.el9.x86 2.2 MB/s | 171 kB 00:00 2024-05-27T08:24:31.860 INFO:teuthology.orchestra.run.smithi092.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3868.g 685 kB/s | 34 kB 00:00 2024-05-27T08:24:31.927 INFO:teuthology.orchestra.run.smithi092.stdout:(29/159): ceph-mgr-cephadm-19.0.0-3868.ga9cb3a5 2.1 MB/s | 143 kB 00:00 2024-05-27T08:24:31.958 INFO:teuthology.orchestra.run.smithi179.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 116 kB/s | 80 kB 00:00 2024-05-27T08:24:32.319 INFO:teuthology.orchestra.run.smithi092.stdout:(30/159): rbd-mirror-19.0.0-3868.ga9cb3a58.el9. 5.1 MB/s | 3.1 MB 00:00 2024-05-27T08:24:32.333 INFO:teuthology.orchestra.run.smithi179.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 783 kB/s | 803 kB 00:01 2024-05-27T08:24:32.653 INFO:teuthology.orchestra.run.smithi092.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3868.ga9cb3 4.9 MB/s | 3.5 MB 00:00 2024-05-27T08:24:32.770 INFO:teuthology.orchestra.run.smithi092.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3868.ga9 2.3 MB/s | 274 kB 00:00 2024-05-27T08:24:32.828 INFO:teuthology.orchestra.run.smithi092.stdout:(33/159): ceph-mgr-rook-19.0.0-3868.ga9cb3a58.e 845 kB/s | 49 kB 00:00 2024-05-27T08:24:32.879 INFO:teuthology.orchestra.run.smithi092.stdout:(34/159): ceph-prometheus-alerts-19.0.0-3868.ga 330 kB/s | 16 kB 00:00 2024-05-27T08:24:32.884 INFO:teuthology.orchestra.run.smithi179.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 211 kB/s | 195 kB 00:00 2024-05-27T08:24:32.979 INFO:teuthology.orchestra.run.smithi092.stdout:(35/159): ceph-volume-19.0.0-3868.ga9cb3a58.el9 2.7 MB/s | 273 kB 00:00 2024-05-27T08:24:33.121 INFO:teuthology.orchestra.run.smithi092.stdout:(36/159): cephadm-19.0.0-3868.ga9cb3a58.el9.noa 5.3 MB/s | 763 kB 00:00 2024-05-27T08:24:33.143 INFO:teuthology.orchestra.run.smithi077.stdout:(8/159): ceph-radosgw-19.0.0-3868.ga9cb3a58.el9 5.6 MB/s | 11 MB 00:01 2024-05-27T08:24:33.179 INFO:teuthology.orchestra.run.smithi092.stdout:(37/159): cephfs-top-19.0.0-3868.ga9cb3a58.el9. 315 kB/s | 18 kB 00:00 2024-05-27T08:24:33.202 INFO:teuthology.orchestra.run.smithi077.stdout:(9/159): ceph-selinux-19.0.0-3868.ga9cb3a58.el9 433 kB/s | 25 kB 00:00 2024-05-27T08:24:33.243 INFO:teuthology.orchestra.run.smithi179.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 279 kB/s | 253 kB 00:00 2024-05-27T08:24:33.613 INFO:teuthology.orchestra.run.smithi092.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-05-27T08:24:33.688 INFO:teuthology.orchestra.run.smithi092.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 158 kB/s | 80 kB 00:00 2024-05-27T08:24:34.068 INFO:teuthology.orchestra.run.smithi179.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 129 kB/s | 106 kB 00:00 2024-05-27T08:24:34.110 INFO:teuthology.orchestra.run.smithi179.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 1.0 MB/s | 1.2 MB 00:01 2024-05-27T08:24:34.347 INFO:teuthology.orchestra.run.smithi092.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 297 kB/s | 195 kB 00:00 2024-05-27T08:24:34.652 INFO:teuthology.orchestra.run.smithi077.stdout:(10/159): ceph-osd-19.0.0-3868.ga9cb3a58.el9.x8 4.0 MB/s | 17 MB 00:04 2024-05-27T08:24:34.736 INFO:teuthology.orchestra.run.smithi077.stdout:(11/159): cephfs-mirror-19.0.0-3868.ga9cb3a58.e 2.6 MB/s | 225 kB 00:00 2024-05-27T08:24:34.803 INFO:teuthology.orchestra.run.smithi077.stdout:(12/159): cephfs-shell-19.0.0-3868.ga9cb3a58.el 406 kB/s | 27 kB 00:00 2024-05-27T08:24:34.827 INFO:teuthology.orchestra.run.smithi179.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 178 kB/s | 135 kB 00:00 2024-05-27T08:24:34.860 INFO:teuthology.orchestra.run.smithi179.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 166 kB/s | 125 kB 00:00 2024-05-27T08:24:34.861 INFO:teuthology.orchestra.run.smithi077.stdout:(13/159): libcephfs-devel-19.0.0-3868.ga9cb3a58 544 kB/s | 32 kB 00:00 2024-05-27T08:24:35.045 INFO:teuthology.orchestra.run.smithi077.stdout:(14/159): libcephfs2-19.0.0-3868.ga9cb3a58.el9. 4.0 MB/s | 747 kB 00:00 2024-05-27T08:24:35.089 INFO:teuthology.orchestra.run.smithi092.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 341 kB/s | 253 kB 00:00 2024-05-27T08:24:35.120 INFO:teuthology.orchestra.run.smithi077.stdout:(15/159): libcephsqlite-19.0.0-3868.ga9cb3a58.e 2.1 MB/s | 164 kB 00:00 2024-05-27T08:24:35.196 INFO:teuthology.orchestra.run.smithi077.stdout:(16/159): librados-devel-19.0.0-3868.ga9cb3a58. 1.6 MB/s | 127 kB 00:00 2024-05-27T08:24:35.330 INFO:teuthology.orchestra.run.smithi077.stdout:(17/159): libradosstriper1-19.0.0-3868.ga9cb3a5 3.7 MB/s | 501 kB 00:00 2024-05-27T08:24:35.902 INFO:teuthology.orchestra.run.smithi179.stdout:(47/159): ceph-test-19.0.0-3868.ga9cb3a58.el9.x 4.8 MB/s | 49 MB 00:10 2024-05-27T08:24:35.936 INFO:teuthology.orchestra.run.smithi179.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 641 kB/s | 689 kB 00:01 2024-05-27T08:24:35.981 INFO:teuthology.orchestra.run.smithi092.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 339 kB/s | 803 kB 00:02 2024-05-27T08:24:36.003 INFO:teuthology.orchestra.run.smithi179.stdout:(49/159): python3-urllib3-1.26.5-5.el9.noarch.r 182 kB/s | 215 kB 00:01 2024-05-27T08:24:36.045 INFO:teuthology.orchestra.run.smithi179.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 983 kB/s | 107 kB 00:00 2024-05-27T08:24:36.154 INFO:teuthology.orchestra.run.smithi179.stdout:(51/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-27T08:24:36.246 INFO:teuthology.orchestra.run.smithi077.stdout:(18/159): librgw2-19.0.0-3868.ga9cb3a58.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-05-27T08:24:36.254 INFO:teuthology.orchestra.run.smithi179.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-05-27T08:24:36.280 INFO:teuthology.orchestra.run.smithi179.stdout:(53/159): elfutils-libelf-devel-0.191-4.el9.x86 79 kB/s | 22 kB 00:00 2024-05-27T08:24:36.313 INFO:teuthology.orchestra.run.smithi077.stdout:(19/159): python3-ceph-argparse-19.0.0-3868.ga9 679 kB/s | 45 kB 00:00 2024-05-27T08:24:36.380 INFO:teuthology.orchestra.run.smithi179.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-05-27T08:24:36.380 INFO:teuthology.orchestra.run.smithi077.stdout:(20/159): python3-ceph-common-19.0.0-3868.ga9cb 2.0 MB/s | 136 kB 00:00 2024-05-27T08:24:36.456 INFO:teuthology.orchestra.run.smithi077.stdout:(21/159): python3-cephfs-19.0.0-3868.ga9cb3a58. 2.1 MB/s | 161 kB 00:00 2024-05-27T08:24:36.481 INFO:teuthology.orchestra.run.smithi179.stdout:(55/159): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-27T08:24:36.481 INFO:teuthology.orchestra.run.smithi092.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 213 kB/s | 106 kB 00:00 2024-05-27T08:24:36.581 INFO:teuthology.orchestra.run.smithi077.stdout:(22/159): python3-rados-19.0.0-3868.ga9cb3a58.e 2.5 MB/s | 321 kB 00:00 2024-05-27T08:24:36.647 INFO:teuthology.orchestra.run.smithi179.stdout:(56/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.6 MB/s | 3.0 MB 00:00 2024-05-27T08:24:36.689 INFO:teuthology.orchestra.run.smithi077.stdout:(23/159): python3-rbd-19.0.0-3868.ga9cb3a58.el9 2.7 MB/s | 300 kB 00:00 2024-05-27T08:24:36.748 INFO:teuthology.orchestra.run.smithi179.stdout:(57/159): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-05-27T08:24:36.756 INFO:teuthology.orchestra.run.smithi077.stdout:(24/159): python3-rgw-19.0.0-3868.ga9cb3a58.el9 1.5 MB/s | 100 kB 00:00 2024-05-27T08:24:36.773 INFO:teuthology.orchestra.run.smithi179.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 548 kB/s | 160 kB 00:00 2024-05-27T08:24:36.798 INFO:teuthology.orchestra.run.smithi179.stdout:(59/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 301 kB/s | 45 kB 00:00 2024-05-27T08:24:36.823 INFO:teuthology.orchestra.run.smithi077.stdout:(25/159): rbd-fuse-19.0.0-3868.ga9cb3a58.el9.x8 1.2 MB/s | 85 kB 00:00 2024-05-27T08:24:36.907 INFO:teuthology.orchestra.run.smithi179.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 235 kB/s | 25 kB 00:00 2024-05-27T08:24:36.932 INFO:teuthology.orchestra.run.smithi179.stdout:(61/159): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-05-27T08:24:36.981 INFO:teuthology.orchestra.run.smithi092.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 270 kB/s | 135 kB 00:00 2024-05-27T08:24:37.016 INFO:teuthology.orchestra.run.smithi179.stdout:(62/159): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-05-27T08:24:37.041 INFO:teuthology.orchestra.run.smithi179.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 353 kB/s | 47 kB 00:00 2024-05-27T08:24:37.098 INFO:teuthology.orchestra.run.smithi092.stdout:(45/159): python3-cryptography-36.0.1-4.el9.x86 625 kB/s | 1.2 MB 00:02 2024-05-27T08:24:37.125 INFO:teuthology.orchestra.run.smithi179.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-27T08:24:37.150 INFO:teuthology.orchestra.run.smithi179.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 324 kB/s | 35 kB 00:00 2024-05-27T08:24:37.191 INFO:teuthology.orchestra.run.smithi077.stdout:(26/159): ceph-common-19.0.0-3868.ga9cb3a58.el9 2.3 MB/s | 21 MB 00:09 2024-05-27T08:24:37.316 INFO:teuthology.orchestra.run.smithi077.stdout:(27/159): rbd-nbd-19.0.0-3868.ga9cb3a58.el9.x86 1.3 MB/s | 171 kB 00:00 2024-05-27T08:24:37.408 INFO:teuthology.orchestra.run.smithi077.stdout:(28/159): ceph-grafana-dashboards-19.0.0-3868.g 373 kB/s | 34 kB 00:00 2024-05-27T08:24:37.409 INFO:teuthology.orchestra.run.smithi179.stdout:(66/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 613 kB/s | 292 kB 00:00 2024-05-27T08:24:37.458 INFO:teuthology.orchestra.run.smithi077.stdout:(29/159): rbd-mirror-19.0.0-3868.ga9cb3a58.el9. 4.9 MB/s | 3.1 MB 00:00 2024-05-27T08:24:37.551 INFO:teuthology.orchestra.run.smithi179.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-27T08:24:37.558 INFO:teuthology.orchestra.run.smithi077.stdout:(30/159): ceph-mgr-cephadm-19.0.0-3868.ga9cb3a5 952 kB/s | 143 kB 00:00 2024-05-27T08:24:37.932 INFO:teuthology.orchestra.run.smithi092.stdout:(46/159): ceph-test-19.0.0-3868.ga9cb3a58.el9.x 5.2 MB/s | 49 MB 00:09 2024-05-27T08:24:37.957 INFO:teuthology.orchestra.run.smithi092.stdout:(47/159): python3-requests-2.25.1-8.el9.noarch. 128 kB/s | 125 kB 00:00 2024-05-27T08:24:37.982 INFO:teuthology.orchestra.run.smithi092.stdout:(48/159): python3-urllib3-1.26.5-5.el9.noarch.r 243 kB/s | 215 kB 00:00 2024-05-27T08:24:38.059 INFO:teuthology.orchestra.run.smithi077.stdout:(31/159): ceph-mgr-dashboard-19.0.0-3868.ga9cb3 5.9 MB/s | 3.5 MB 00:00 2024-05-27T08:24:38.125 INFO:teuthology.orchestra.run.smithi077.stdout:(32/159): ceph-mgr-modules-core-19.0.0-3868.ga9 4.0 MB/s | 274 kB 00:00 2024-05-27T08:24:38.192 INFO:teuthology.orchestra.run.smithi077.stdout:(33/159): ceph-mgr-rook-19.0.0-3868.ga9cb3a58.e 739 kB/s | 49 kB 00:00 2024-05-27T08:24:38.635 INFO:teuthology.orchestra.run.smithi077.stdout:(34/159): ceph-test-19.0.0-3868.ga9cb3a58.el9.x 9.0 MB/s | 49 MB 00:05 2024-05-27T08:24:38.660 INFO:teuthology.orchestra.run.smithi077.stdout:(35/159): ceph-prometheus-alerts-19.0.0-3868.ga 35 kB/s | 16 kB 00:00 2024-05-27T08:24:38.727 INFO:teuthology.orchestra.run.smithi077.stdout:(36/159): ceph-volume-19.0.0-3868.ga9cb3a58.el9 2.9 MB/s | 273 kB 00:00 2024-05-27T08:24:38.749 INFO:teuthology.orchestra.run.smithi092.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 139 kB/s | 107 kB 00:00 2024-05-27T08:24:38.777 INFO:teuthology.orchestra.run.smithi077.stdout:(37/159): cephfs-top-19.0.0-3868.ga9cb3a58.el9. 366 kB/s | 18 kB 00:00 2024-05-27T08:24:38.844 INFO:teuthology.orchestra.run.smithi077.stdout:(38/159): cephadm-19.0.0-3868.ga9cb3a58.el9.noa 4.1 MB/s | 763 kB 00:00 2024-05-27T08:24:38.886 INFO:teuthology.orchestra.run.smithi077.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-05-27T08:24:38.916 INFO:teuthology.orchestra.run.smithi092.stdout:(50/159): automake-1.16.2-8.el9.noarch.rpm 700 kB/s | 689 kB 00:00 2024-05-27T08:24:38.935 INFO:teuthology.orchestra.run.smithi179.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 658 kB/s | 911 kB 00:01 2024-05-27T08:24:38.961 INFO:teuthology.orchestra.run.smithi077.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-05-27T08:24:38.993 INFO:teuthology.orchestra.run.smithi179.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-05-27T08:24:39.008 INFO:teuthology.orchestra.run.smithi092.stdout:(51/159): bison-3.7.4-5.el9.x86_64.rpm 898 kB/s | 943 kB 00:01 2024-05-27T08:24:39.012 INFO:teuthology.orchestra.run.smithi077.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-27T08:24:39.046 INFO:teuthology.orchestra.run.smithi077.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-05-27T08:24:39.087 INFO:teuthology.orchestra.run.smithi077.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-27T08:24:39.121 INFO:teuthology.orchestra.run.smithi077.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-05-27T08:24:39.138 INFO:teuthology.orchestra.run.smithi077.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-05-27T08:24:39.163 INFO:teuthology.orchestra.run.smithi077.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-05-27T08:24:39.189 INFO:teuthology.orchestra.run.smithi077.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-27T08:24:39.201 INFO:teuthology.orchestra.run.smithi179.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 501 kB/s | 133 kB 00:00 2024-05-27T08:24:39.306 INFO:teuthology.orchestra.run.smithi077.stdout:(48/159): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-05-27T08:24:39.391 INFO:teuthology.orchestra.run.smithi092.stdout:(52/159): elfutils-libelf-devel-0.191-4.el9.x86 34 kB/s | 22 kB 00:00 2024-05-27T08:24:39.456 INFO:teuthology.orchestra.run.smithi077.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 712 kB/s | 107 kB 00:00 2024-05-27T08:24:39.498 INFO:teuthology.orchestra.run.smithi077.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-05-27T08:24:39.532 INFO:teuthology.orchestra.run.smithi077.stdout:(51/159): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-05-27T08:24:39.582 INFO:teuthology.orchestra.run.smithi077.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-05-27T08:24:39.607 INFO:teuthology.orchestra.run.smithi077.stdout:(53/159): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-27T08:24:39.675 INFO:teuthology.orchestra.run.smithi092.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 51 kB/s | 33 kB 00:00 2024-05-27T08:24:39.691 INFO:teuthology.orchestra.run.smithi077.stdout:(54/159): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 943 kB 00:00 2024-05-27T08:24:39.717 INFO:teuthology.orchestra.run.smithi077.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 167 kB/s | 18 kB 00:00 2024-05-27T08:24:39.742 INFO:teuthology.orchestra.run.smithi077.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-27T08:24:39.784 INFO:teuthology.orchestra.run.smithi077.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-05-27T08:24:39.800 INFO:teuthology.orchestra.run.smithi092.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 355 kB/s | 313 kB 00:00 2024-05-27T08:24:39.817 INFO:teuthology.orchestra.run.smithi077.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-05-27T08:24:39.868 INFO:teuthology.orchestra.run.smithi077.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-05-27T08:24:39.909 INFO:teuthology.orchestra.run.smithi077.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 614 kB/s | 25 kB 00:00 2024-05-27T08:24:39.952 INFO:teuthology.orchestra.run.smithi077.stdout:(61/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-05-27T08:24:39.985 INFO:teuthology.orchestra.run.smithi077.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-05-27T08:24:40.002 INFO:teuthology.orchestra.run.smithi179.stdout:(71/159): openblas-openmp-0.3.21-2.el9.x86_64.r 1.6 MB/s | 4.7 MB 00:02 2024-05-27T08:24:40.015 INFO:teuthology.orchestra.run.smithi077.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-05-27T08:24:40.086 INFO:teuthology.orchestra.run.smithi077.stdout:(64/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.9 MB/s | 3.0 MB 00:00 2024-05-27T08:24:40.111 INFO:teuthology.orchestra.run.smithi077.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-27T08:24:40.136 INFO:teuthology.orchestra.run.smithi077.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 280 kB/s | 35 kB 00:00 2024-05-27T08:24:40.192 INFO:teuthology.orchestra.run.smithi092.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-05-27T08:24:40.194 INFO:teuthology.orchestra.run.smithi179.stdout:(72/159): python3-attrs-20.3.0-7.el9.noarch.rpm 455 kB/s | 87 kB 00:00 2024-05-27T08:24:40.229 INFO:teuthology.orchestra.run.smithi077.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-27T08:24:40.287 INFO:teuthology.orchestra.run.smithi077.stdout:(68/159): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-05-27T08:24:40.309 INFO:teuthology.orchestra.run.smithi092.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-05-27T08:24:40.329 INFO:teuthology.orchestra.run.smithi077.stdout:(69/159): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-05-27T08:24:40.396 INFO:teuthology.orchestra.run.smithi077.stdout:(70/159): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-05-27T08:24:40.505 INFO:teuthology.orchestra.run.smithi077.stdout:(71/159): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-05-27T08:24:40.518 INFO:teuthology.orchestra.run.smithi092.stdout:(57/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-05-27T08:24:40.547 INFO:teuthology.orchestra.run.smithi077.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-05-27T08:24:40.572 INFO:teuthology.orchestra.run.smithi077.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-27T08:24:40.630 INFO:teuthology.orchestra.run.smithi077.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-05-27T08:24:40.636 INFO:teuthology.orchestra.run.smithi179.stdout:(73/159): protobuf-3.14.0-13.el9.x86_64.rpm 720 kB/s | 1.0 MB 00:01 2024-05-27T08:24:40.681 INFO:teuthology.orchestra.run.smithi077.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-27T08:24:40.739 INFO:teuthology.orchestra.run.smithi077.stdout:(76/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-05-27T08:24:40.745 INFO:teuthology.orchestra.run.smithi179.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-05-27T08:24:40.765 INFO:teuthology.orchestra.run.smithi077.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-27T08:24:40.790 INFO:teuthology.orchestra.run.smithi077.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-05-27T08:24:40.815 INFO:teuthology.orchestra.run.smithi077.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-05-27T08:24:40.891 INFO:teuthology.orchestra.run.smithi077.stdout:(80/159): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-27T08:24:40.916 INFO:teuthology.orchestra.run.smithi077.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 346 kB/s | 35 kB 00:00 2024-05-27T08:24:40.926 INFO:teuthology.orchestra.run.smithi092.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 73 kB/s | 45 kB 00:00 2024-05-27T08:24:40.941 INFO:teuthology.orchestra.run.smithi077.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-05-27T08:24:40.966 INFO:teuthology.orchestra.run.smithi077.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-27T08:24:40.968 INFO:teuthology.orchestra.run.smithi092.stdout:(59/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 206 kB/s | 160 kB 00:00 2024-05-27T08:24:41.000 INFO:teuthology.orchestra.run.smithi077.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-27T08:24:41.026 INFO:teuthology.orchestra.run.smithi077.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-27T08:24:41.051 INFO:teuthology.orchestra.run.smithi077.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-05-27T08:24:41.076 INFO:teuthology.orchestra.run.smithi077.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-05-27T08:24:41.087 INFO:teuthology.orchestra.run.smithi179.stdout:(75/159): python3-devel-3.9.18-3.el9.x86_64.rpm 714 kB/s | 244 kB 00:00 2024-05-27T08:24:41.152 INFO:teuthology.orchestra.run.smithi077.stdout:(88/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-05-27T08:24:41.177 INFO:teuthology.orchestra.run.smithi077.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-05-27T08:24:41.202 INFO:teuthology.orchestra.run.smithi077.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-27T08:24:41.227 INFO:teuthology.orchestra.run.smithi077.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-27T08:24:41.253 INFO:teuthology.orchestra.run.smithi077.stdout:(92/159): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-05-27T08:24:41.278 INFO:teuthology.orchestra.run.smithi077.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 828 kB/s | 42 kB 00:00 2024-05-27T08:24:41.303 INFO:teuthology.orchestra.run.smithi077.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-05-27T08:24:41.337 INFO:teuthology.orchestra.run.smithi077.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-27T08:24:41.354 INFO:teuthology.orchestra.run.smithi077.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 711 kB/s | 36 kB 00:00 2024-05-27T08:24:41.372 INFO:teuthology.orchestra.run.smithi077.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-05-27T08:24:41.405 INFO:teuthology.orchestra.run.smithi077.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-05-27T08:24:41.421 INFO:teuthology.orchestra.run.smithi077.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-27T08:24:41.438 INFO:teuthology.orchestra.run.smithi179.stdout:(76/159): python3-jinja2-2.11.3-6.el9.noarch.rp 709 kB/s | 248 kB 00:00 2024-05-27T08:24:41.443 INFO:teuthology.orchestra.run.smithi092.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 716 kB/s | 662 kB 00:00 2024-05-27T08:24:41.447 INFO:teuthology.orchestra.run.smithi077.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-05-27T08:24:41.514 INFO:teuthology.orchestra.run.smithi077.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 691 kB/s | 64 kB 00:00 2024-05-27T08:24:41.535 INFO:teuthology.orchestra.run.smithi092.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 45 kB/s | 25 kB 00:00 2024-05-27T08:24:41.537 INFO:teuthology.orchestra.run.smithi179.stdout:(77/159): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-27T08:24:41.539 INFO:teuthology.orchestra.run.smithi077.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 487 kB/s | 45 kB 00:00 2024-05-27T08:24:41.638 INFO:teuthology.orchestra.run.smithi179.stdout:(78/159): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-05-27T08:24:41.740 INFO:teuthology.orchestra.run.smithi077.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-05-27T08:24:41.765 INFO:teuthology.orchestra.run.smithi077.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 89 kB/s | 22 kB 00:00 2024-05-27T08:24:41.827 INFO:teuthology.orchestra.run.smithi092.stdout:(62/159): libtool-2.4.6-45.el9.x86_64.rpm 647 kB/s | 582 kB 00:00 2024-05-27T08:24:41.897 INFO:teuthology.orchestra.run.smithi179.stdout:(79/159): python3-mako-1.1.4-6.el9.noarch.rpm 666 kB/s | 172 kB 00:00 2024-05-27T08:24:41.924 INFO:teuthology.orchestra.run.smithi077.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-05-27T08:24:42.005 INFO:teuthology.orchestra.run.smithi179.stdout:(80/159): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-05-27T08:24:42.061 INFO:teuthology.orchestra.run.smithi092.stdout:(63/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 77 kB/s | 47 kB 00:00 2024-05-27T08:24:42.157 INFO:teuthology.orchestra.run.smithi077.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 788 kB/s | 309 kB 00:00 2024-05-27T08:24:42.182 INFO:teuthology.orchestra.run.smithi077.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-05-27T08:24:42.274 INFO:teuthology.orchestra.run.smithi077.stdout:(108/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-05-27T08:24:42.300 INFO:teuthology.orchestra.run.smithi077.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-05-27T08:24:42.350 INFO:teuthology.orchestra.run.smithi077.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-05-27T08:24:42.386 INFO:teuthology.orchestra.run.smithi092.stdout:(64/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 344 kB/s | 292 kB 00:00 2024-05-27T08:24:42.467 INFO:teuthology.orchestra.run.smithi077.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-27T08:24:42.492 INFO:teuthology.orchestra.run.smithi077.stdout:(112/159): libunwind-1.6.2-1.el9.x86_64.rpm 350 kB/s | 67 kB 00:00 2024-05-27T08:24:42.517 INFO:teuthology.orchestra.run.smithi077.stdout:(113/159): luarocks-3.9.2-5.el9.noarch.rpm 905 kB/s | 151 kB 00:00 2024-05-27T08:24:42.559 INFO:teuthology.orchestra.run.smithi077.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-05-27T08:24:42.593 INFO:teuthology.orchestra.run.smithi077.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 577 kB/s | 43 kB 00:00 2024-05-27T08:24:42.627 INFO:teuthology.orchestra.run.smithi077.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-05-27T08:24:42.637 INFO:teuthology.orchestra.run.smithi092.stdout:(65/159): lua-5.4.4-4.el9.x86_64.rpm 233 kB/s | 188 kB 00:00 2024-05-27T08:24:42.660 INFO:teuthology.orchestra.run.smithi077.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-27T08:24:42.687 INFO:teuthology.orchestra.run.smithi092.stdout:(66/159): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-05-27T08:24:42.702 INFO:teuthology.orchestra.run.smithi077.stdout:(118/159): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-05-27T08:24:42.727 INFO:teuthology.orchestra.run.smithi077.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-05-27T08:24:42.769 INFO:teuthology.orchestra.run.smithi077.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-05-27T08:24:42.795 INFO:teuthology.orchestra.run.smithi077.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-27T08:24:42.820 INFO:teuthology.orchestra.run.smithi077.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-27T08:24:42.879 INFO:teuthology.orchestra.run.smithi077.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-05-27T08:24:42.904 INFO:teuthology.orchestra.run.smithi077.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-05-27T08:24:42.946 INFO:teuthology.orchestra.run.smithi077.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-05-27T08:24:42.972 INFO:teuthology.orchestra.run.smithi077.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-05-27T08:24:43.022 INFO:teuthology.orchestra.run.smithi077.stdout:(127/159): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-05-27T08:24:43.047 INFO:teuthology.orchestra.run.smithi077.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-05-27T08:24:43.064 INFO:teuthology.orchestra.run.smithi077.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-05-27T08:24:43.123 INFO:teuthology.orchestra.run.smithi077.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-05-27T08:24:43.148 INFO:teuthology.orchestra.run.smithi077.stdout:(131/159): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-05-27T08:24:43.173 INFO:teuthology.orchestra.run.smithi077.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 725 kB/s | 79 kB 00:00 2024-05-27T08:24:43.198 INFO:teuthology.orchestra.run.smithi077.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-05-27T08:24:43.224 INFO:teuthology.orchestra.run.smithi077.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-27T08:24:43.241 INFO:teuthology.orchestra.run.smithi077.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-27T08:24:43.266 INFO:teuthology.orchestra.run.smithi077.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-27T08:24:43.283 INFO:teuthology.orchestra.run.smithi077.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-05-27T08:24:43.300 INFO:teuthology.orchestra.run.smithi077.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 522 kB/s | 31 kB 00:00 2024-05-27T08:24:43.342 INFO:teuthology.orchestra.run.smithi077.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-27T08:24:43.367 INFO:teuthology.orchestra.run.smithi077.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 704 kB/s | 59 kB 00:00 2024-05-27T08:24:43.392 INFO:teuthology.orchestra.run.smithi077.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-05-27T08:24:43.418 INFO:teuthology.orchestra.run.smithi077.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 674 kB/s | 51 kB 00:00 2024-05-27T08:24:43.451 INFO:teuthology.orchestra.run.smithi077.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-27T08:24:43.477 INFO:teuthology.orchestra.run.smithi077.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-27T08:24:43.510 INFO:teuthology.orchestra.run.smithi077.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-05-27T08:24:43.535 INFO:teuthology.orchestra.run.smithi077.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-05-27T08:24:43.569 INFO:teuthology.orchestra.run.smithi077.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-27T08:24:43.603 INFO:teuthology.orchestra.run.smithi092.stdout:(67/159): openblas-openmp-0.3.21-2.el9.x86_64.r 3.9 MB/s | 4.7 MB 00:01 2024-05-27T08:24:43.644 INFO:teuthology.orchestra.run.smithi077.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-27T08:24:43.787 INFO:teuthology.orchestra.run.smithi092.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-05-27T08:24:43.820 INFO:teuthology.orchestra.run.smithi077.stdout:(149/159): NetworkManager-1.47.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-05-27T08:24:43.887 INFO:teuthology.orchestra.run.smithi077.stdout:(150/159): NetworkManager-libnm-1.47.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-05-27T08:24:43.912 INFO:teuthology.orchestra.run.smithi092.stdout:(69/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-05-27T08:24:43.929 INFO:teuthology.orchestra.run.smithi077.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 992 kB/s | 41 kB 00:00 2024-05-27T08:24:43.971 INFO:teuthology.orchestra.run.smithi077.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-05-27T08:24:44.021 INFO:teuthology.orchestra.run.smithi077.stdout:(153/159): librbd1-19.0.0-3868.ga9cb3a58.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-05-27T08:24:44.046 INFO:teuthology.orchestra.run.smithi077.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 489 kB/s | 37 kB 00:00 2024-05-27T08:24:44.088 INFO:teuthology.orchestra.run.smithi077.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 6.1 MB/s | 260 kB 00:00 2024-05-27T08:24:44.147 INFO:teuthology.orchestra.run.smithi077.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-05-27T08:24:44.172 INFO:teuthology.orchestra.run.smithi077.stdout:(157/159): elfutils-libelf-0.191-4.el9.x86_64.r 1.3 MB/s | 207 kB 00:00 2024-05-27T08:24:44.248 INFO:teuthology.orchestra.run.smithi077.stdout:(158/159): librados2-19.0.0-3868.ga9cb3a58.el9. 4.8 MB/s | 3.4 MB 00:00 2024-05-27T08:24:44.289 INFO:teuthology.orchestra.run.smithi077.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-05-27T08:24:44.296 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-05-27T08:24:44.296 INFO:teuthology.orchestra.run.smithi077.stdout:Total 14 MB/s | 227 MB 00:16 2024-05-27T08:24:44.596 INFO:teuthology.orchestra.run.smithi092.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 165 kB/s | 133 kB 00:00 2024-05-27T08:24:44.679 INFO:teuthology.orchestra.run.smithi092.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 847 kB/s | 911 kB 00:01 2024-05-27T08:24:45.229 INFO:teuthology.orchestra.run.smithi092.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-05-27T08:24:45.430 INFO:teuthology.orchestra.run.smithi092.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 117 kB/s | 87 kB 00:00 2024-05-27T08:24:45.508 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-05-27T08:24:45.630 INFO:teuthology.orchestra.run.smithi092.stdout:(74/159): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-05-27T08:24:45.728 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-05-27T08:24:45.728 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-05-27T08:24:46.081 INFO:teuthology.orchestra.run.smithi092.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 94 kB/s | 61 kB 00:00 2024-05-27T08:24:46.431 INFO:teuthology.orchestra.run.smithi092.stdout:(76/159): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-05-27T08:24:46.460 INFO:teuthology.orchestra.run.smithi092.stdout:(77/159): python3-devel-3.9.18-3.el9.x86_64.rpm 296 kB/s | 244 kB 00:00 2024-05-27T08:24:46.914 INFO:teuthology.orchestra.run.smithi092.stdout:(78/159): python3-jinja2-2.11.3-6.el9.noarch.rp 298 kB/s | 248 kB 00:00 2024-05-27T08:24:46.987 INFO:teuthology.orchestra.run.smithi092.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-05-27T08:24:47.005 INFO:teuthology.orchestra.run.smithi179.stdout:(81/159): postgresql-server-13.14-1.el9.x86_64. 739 kB/s | 5.8 MB 00:08 2024-05-27T08:24:47.006 INFO:teuthology.orchestra.run.smithi092.stdout:(80/159): python3-jsonpatch-1.21-16.el9.noarch. 45 kB/s | 26 kB 00:00 2024-05-27T08:24:47.189 INFO:teuthology.orchestra.run.smithi179.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-05-27T08:24:47.298 INFO:teuthology.orchestra.run.smithi179.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-05-27T08:24:47.423 INFO:teuthology.orchestra.run.smithi179.stdout:(84/159): python3-packaging-20.9-5.el9.noarch.r 617 kB/s | 77 kB 00:00 2024-05-27T08:24:47.531 INFO:teuthology.orchestra.run.smithi179.stdout:(85/159): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-05-27T08:24:47.581 INFO:teuthology.orchestra.run.smithi092.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 58 kB/s | 35 kB 00:00 2024-05-27T08:24:47.641 INFO:teuthology.orchestra.run.smithi179.stdout:(86/159): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-05-27T08:24:47.723 INFO:teuthology.orchestra.run.smithi092.stdout:(82/159): python3-mako-1.1.4-6.el9.noarch.rpm 213 kB/s | 172 kB 00:00 2024-05-27T08:24:47.749 INFO:teuthology.orchestra.run.smithi179.stdout:(87/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-05-27T08:24:47.858 INFO:teuthology.orchestra.run.smithi179.stdout:(88/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-27T08:24:47.894 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-05-27T08:24:47.894 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-05-27T08:24:47.966 INFO:teuthology.orchestra.run.smithi179.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-05-27T08:24:48.075 INFO:teuthology.orchestra.run.smithi179.stdout:(90/159): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-05-27T08:24:48.167 INFO:teuthology.orchestra.run.smithi179.stdout:(91/159): python3-babel-2.9.1-2.el9.noarch.rpm 766 kB/s | 6.0 MB 00:07 2024-05-27T08:24:48.249 INFO:teuthology.orchestra.run.smithi092.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-05-27T08:24:48.267 INFO:teuthology.orchestra.run.smithi179.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-27T08:24:48.368 INFO:teuthology.orchestra.run.smithi179.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-05-27T08:24:48.382 INFO:teuthology.orchestra.run.smithi092.stdout:(84/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 313 kB/s | 250 kB 00:00 2024-05-27T08:24:48.566 INFO:teuthology.orchestra.run.smithi092.stdout:(85/159): python3-oauthlib-3.1.1-5.el9.noarch.r 264 kB/s | 222 kB 00:00 2024-05-27T08:24:48.711 INFO:teuthology.orchestra.run.smithi179.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 646 kB/s | 221 kB 00:00 2024-05-27T08:24:48.810 INFO:teuthology.orchestra.run.smithi179.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-05-27T08:24:48.902 INFO:teuthology.orchestra.run.smithi179.stdout:(96/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 516 kB/s | 47 kB 00:00 2024-05-27T08:24:48.933 INFO:teuthology.orchestra.run.smithi092.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 113 kB/s | 77 kB 00:00 2024-05-27T08:24:48.986 INFO:teuthology.orchestra.run.smithi179.stdout:(97/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 748 kB/s | 5.1 MB 00:06 2024-05-27T08:24:48.991 INFO:teuthology.orchestra.run.smithi092.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 69 kB/s | 42 kB 00:00 2024-05-27T08:24:49.094 INFO:teuthology.orchestra.run.smithi179.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-05-27T08:24:49.286 INFO:teuthology.orchestra.run.smithi179.stdout:(99/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 489 kB/s | 94 kB 00:00 2024-05-27T08:24:49.328 INFO:teuthology.orchestra.run.smithi179.stdout:(100/159): socat-1.7.4.1-5.el9.x86_64.rpm 715 kB/s | 305 kB 00:00 2024-05-27T08:24:49.375 INFO:teuthology.orchestra.run.smithi092.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 333 kB/s | 269 kB 00:00 2024-05-27T08:24:49.437 INFO:teuthology.orchestra.run.smithi179.stdout:(101/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-05-27T08:24:49.471 INFO:teuthology.orchestra.run.smithi179.stdout:(102/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 346 kB/s | 64 kB 00:00 2024-05-27T08:24:49.538 INFO:teuthology.orchestra.run.smithi179.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-27T08:24:49.692 INFO:teuthology.orchestra.run.smithi092.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 209 kB/s | 159 kB 00:00 2024-05-27T08:24:49.705 INFO:teuthology.orchestra.run.smithi179.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-05-27T08:24:49.817 INFO:teuthology.orchestra.run.smithi092.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 338 kB/s | 279 kB 00:00 2024-05-27T08:24:50.009 INFO:teuthology.orchestra.run.smithi092.stdout:(91/159): python3-pytz-2021.1-5.el9.noarch.rpm 80 kB/s | 51 kB 00:00 2024-05-27T08:24:50.155 INFO:teuthology.orchestra.run.smithi179.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 888 kB/s | 548 kB 00:00 2024-05-27T08:24:50.180 INFO:teuthology.orchestra.run.smithi179.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 651 kB/s | 309 kB 00:00 2024-05-27T08:24:50.206 INFO:teuthology.orchestra.run.smithi179.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 386 kB/s | 19 kB 00:00 2024-05-27T08:24:50.264 INFO:teuthology.orchestra.run.smithi179.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-05-27T08:24:50.315 INFO:teuthology.orchestra.run.smithi179.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2024-05-27T08:24:50.342 INFO:teuthology.orchestra.run.smithi092.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 82 kB/s | 54 kB 00:00 2024-05-27T08:24:50.374 INFO:teuthology.orchestra.run.smithi179.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-27T08:24:50.566 INFO:teuthology.orchestra.run.smithi179.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 790 kB/s | 151 kB 00:00 2024-05-27T08:24:50.626 INFO:teuthology.orchestra.run.smithi092.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-05-27T08:24:50.633 INFO:teuthology.orchestra.run.smithi179.stdout:(112/159): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-05-27T08:24:50.699 INFO:teuthology.orchestra.run.smithi179.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-27T08:24:50.725 INFO:teuthology.orchestra.run.smithi179.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-05-27T08:24:50.758 INFO:teuthology.orchestra.run.smithi179.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-05-27T08:24:50.784 INFO:teuthology.orchestra.run.smithi179.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 548 kB/s | 32 kB 00:00 2024-05-27T08:24:50.817 INFO:teuthology.orchestra.run.smithi179.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-05-27T08:24:50.852 INFO:teuthology.orchestra.run.smithi179.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-27T08:24:50.885 INFO:teuthology.orchestra.run.smithi179.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-05-27T08:24:50.910 INFO:teuthology.orchestra.run.smithi179.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-05-27T08:24:50.960 INFO:teuthology.orchestra.run.smithi092.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 72 kB/s | 44 kB 00:00 2024-05-27T08:24:50.962 INFO:teuthology.orchestra.run.smithi179.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-27T08:24:51.028 INFO:teuthology.orchestra.run.smithi179.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-27T08:24:51.120 INFO:teuthology.orchestra.run.smithi179.stdout:(123/159): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-05-27T08:24:51.145 INFO:teuthology.orchestra.run.smithi179.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-05-27T08:24:51.187 INFO:teuthology.orchestra.run.smithi179.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 263 kB/s | 18 kB 00:00 2024-05-27T08:24:51.213 INFO:teuthology.orchestra.run.smithi179.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-05-27T08:24:51.246 INFO:teuthology.orchestra.run.smithi179.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-05-27T08:24:51.272 INFO:teuthology.orchestra.run.smithi179.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-05-27T08:24:51.372 INFO:teuthology.orchestra.run.smithi179.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-05-27T08:24:51.376 INFO:teuthology.orchestra.run.smithi092.stdout:(95/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-27T08:24:51.406 INFO:teuthology.orchestra.run.smithi179.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-05-27T08:24:51.431 INFO:teuthology.orchestra.run.smithi179.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-27T08:24:51.473 INFO:teuthology.orchestra.run.smithi179.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 860 kB/s | 58 kB 00:00 2024-05-27T08:24:51.502 INFO:teuthology.orchestra.run.smithi092.stdout:(96/159): qatlib-24.02.0-1.el9.x86_64.rpm 252 kB/s | 221 kB 00:00 2024-05-27T08:24:51.506 INFO:teuthology.orchestra.run.smithi179.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-27T08:24:51.540 INFO:teuthology.orchestra.run.smithi179.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-27T08:24:51.565 INFO:teuthology.orchestra.run.smithi179.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-27T08:24:51.607 INFO:teuthology.orchestra.run.smithi179.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-05-27T08:24:51.641 INFO:teuthology.orchestra.run.smithi179.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-27T08:24:51.652 INFO:teuthology.orchestra.run.smithi092.stdout:(97/159): qatlib-service-24.02.0-1.el9.x86_64.r 52 kB/s | 36 kB 00:00 2024-05-27T08:24:51.683 INFO:teuthology.orchestra.run.smithi179.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-27T08:24:51.709 INFO:teuthology.orchestra.run.smithi179.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 880 kB/s | 59 kB 00:00 2024-05-27T08:24:51.742 INFO:teuthology.orchestra.run.smithi179.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-05-27T08:24:51.776 INFO:teuthology.orchestra.run.smithi179.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 760 kB/s | 51 kB 00:00 2024-05-27T08:24:51.810 INFO:teuthology.orchestra.run.smithi179.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-05-27T08:24:51.827 INFO:teuthology.orchestra.run.smithi179.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-27T08:24:51.877 INFO:teuthology.orchestra.run.smithi179.stdout:(144/159): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-05-27T08:24:51.894 INFO:teuthology.orchestra.run.smithi179.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-05-27T08:24:51.962 INFO:teuthology.orchestra.run.smithi179.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-27T08:24:52.002 INFO:teuthology.orchestra.run.smithi092.stdout:(98/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-05-27T08:24:52.062 INFO:teuthology.orchestra.run.smithi179.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-27T08:24:52.327 INFO:teuthology.orchestra.run.smithi092.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 369 kB/s | 305 kB 00:00 2024-05-27T08:24:52.344 INFO:teuthology.orchestra.run.smithi092.stdout:(100/159): uuid-1.6.2-55.el9.x86_64.rpm 83 kB/s | 58 kB 00:00 2024-05-27T08:24:52.570 INFO:teuthology.orchestra.run.smithi179.stdout:(148/159): librados2-19.0.0-3868.ga9cb3a58.el9. 5.6 MB/s | 3.4 MB 00:00 2024-05-27T08:24:52.745 INFO:teuthology.orchestra.run.smithi092.stdout:(101/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 126 kB/s | 94 kB 00:00 2024-05-27T08:24:52.803 INFO:teuthology.orchestra.run.smithi092.stdout:(102/159): lua-devel-5.4.4-4.el9.x86_64.rpm 386 kB/s | 22 kB 00:00 2024-05-27T08:24:52.904 INFO:teuthology.orchestra.run.smithi179.stdout:(149/159): librbd1-19.0.0-3868.ga9cb3a58.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-05-27T08:24:52.928 INFO:teuthology.orchestra.run.smithi092.stdout:(103/159): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-05-27T08:24:52.978 INFO:teuthology.orchestra.run.smithi092.stdout:(104/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-05-27T08:24:53.004 INFO:teuthology.orchestra.run.smithi092.stdout:(105/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-05-27T08:24:53.171 INFO:teuthology.orchestra.run.smithi092.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 117 kB/s | 19 kB 00:00 2024-05-27T08:24:53.313 INFO:teuthology.orchestra.run.smithi092.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-05-27T08:24:53.355 INFO:teuthology.orchestra.run.smithi092.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 822 kB/s | 309 kB 00:00 2024-05-27T08:24:53.405 INFO:teuthology.orchestra.run.smithi092.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-05-27T08:24:53.439 INFO:teuthology.orchestra.run.smithi092.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 581 kB/s | 49 kB 00:00 2024-05-27T08:24:53.480 INFO:teuthology.orchestra.run.smithi092.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 896 kB/s | 67 kB 00:00 2024-05-27T08:24:53.522 INFO:teuthology.orchestra.run.smithi092.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-05-27T08:24:53.631 INFO:teuthology.orchestra.run.smithi092.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 839 kB 00:00 2024-05-27T08:24:53.656 INFO:teuthology.orchestra.run.smithi092.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-05-27T08:24:53.662 INFO:teuthology.orchestra.run.smithi179.stdout:(150/159): NetworkManager-1.47.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-05-27T08:24:53.715 INFO:teuthology.orchestra.run.smithi092.stdout:(115/159): libarrow-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2024-05-27T08:24:53.740 INFO:teuthology.orchestra.run.smithi092.stdout:(116/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 399 kB/s | 43 kB 00:00 2024-05-27T08:24:53.765 INFO:teuthology.orchestra.run.smithi092.stdout:(117/159): python3-cachetools-4.2.4-1.el9.noarc 295 kB/s | 32 kB 00:00 2024-05-27T08:24:53.816 INFO:teuthology.orchestra.run.smithi092.stdout:(118/159): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-27T08:24:53.858 INFO:teuthology.orchestra.run.smithi092.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-27T08:24:53.891 INFO:teuthology.orchestra.run.smithi092.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-27T08:24:53.917 INFO:teuthology.orchestra.run.smithi092.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-05-27T08:24:53.942 INFO:teuthology.orchestra.run.smithi092.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-27T08:24:53.954 INFO:teuthology.orchestra.run.smithi179.stdout:(151/159): NetworkManager-libnm-1.47.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-05-27T08:24:54.017 INFO:teuthology.orchestra.run.smithi092.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-27T08:24:54.042 INFO:teuthology.orchestra.run.smithi092.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-05-27T08:24:54.093 INFO:teuthology.orchestra.run.smithi092.stdout:(125/159): python3-grpcio-1.46.7-10.el9.x86_64. 10 MB/s | 2.0 MB 00:00 2024-05-27T08:24:54.118 INFO:teuthology.orchestra.run.smithi092.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-27T08:24:54.143 INFO:teuthology.orchestra.run.smithi092.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-27T08:24:54.177 INFO:teuthology.orchestra.run.smithi092.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-05-27T08:24:54.202 INFO:teuthology.orchestra.run.smithi092.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2024-05-27T08:24:54.253 INFO:teuthology.orchestra.run.smithi092.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-05-27T08:24:54.259 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-05-27T08:24:54.280 INFO:teuthology.orchestra.run.smithi179.stdout:(152/159): NetworkManager-team-1.47.90-1.el9.x8 67 kB/s | 41 kB 00:00 2024-05-27T08:24:54.286 INFO:teuthology.orchestra.run.smithi092.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-05-27T08:24:54.311 INFO:teuthology.orchestra.run.smithi092.stdout:(132/159): python3-more-itertools-8.12.0-2.el9. 724 kB/s | 79 kB 00:00 2024-05-27T08:24:54.336 INFO:teuthology.orchestra.run.smithi092.stdout:(133/159): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-05-27T08:24:54.345 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T08:24:54.387 INFO:teuthology.orchestra.run.smithi092.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-05-27T08:24:54.391 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T08:24:54.412 INFO:teuthology.orchestra.run.smithi092.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-05-27T08:24:54.429 INFO:teuthology.orchestra.run.smithi092.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-05-27T08:24:54.445 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T08:24:54.478 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T08:24:54.488 INFO:teuthology.orchestra.run.smithi092.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-05-27T08:24:54.504 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T08:24:54.513 INFO:teuthology.orchestra.run.smithi092.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-27T08:24:54.538 INFO:teuthology.orchestra.run.smithi092.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-27T08:24:54.559 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T08:24:54.563 INFO:teuthology.orchestra.run.smithi092.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-05-27T08:24:54.588 INFO:teuthology.orchestra.run.smithi092.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-27T08:24:54.622 INFO:teuthology.orchestra.run.smithi092.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 607 kB/s | 51 kB 00:00 2024-05-27T08:24:54.656 INFO:teuthology.orchestra.run.smithi092.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-27T08:24:54.681 INFO:teuthology.orchestra.run.smithi092.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 972 kB/s | 90 kB 00:00 2024-05-27T08:24:54.732 INFO:teuthology.orchestra.run.smithi092.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-27T08:24:54.755 INFO:teuthology.orchestra.run.smithi179.stdout:(153/159): NetworkManager-tui-1.47.90-1.el9.x86 310 kB/s | 248 kB 00:00 2024-05-27T08:24:54.773 INFO:teuthology.orchestra.run.smithi092.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-05-27T08:24:54.799 INFO:teuthology.orchestra.run.smithi092.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-27T08:24:54.892 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T08:24:54.897 INFO:teuthology.orchestra.run.smithi179.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 60 kB/s | 37 kB 00:00 2024-05-27T08:24:54.899 INFO:teuthology.orchestra.run.smithi092.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-27T08:24:55.005 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:24:55.126 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:24:55.156 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:24:55.208 INFO:teuthology.orchestra.run.smithi092.stdout:(149/159): librbd1-19.0.0-3868.ga9cb3a58.el9.x8 7.7 MB/s | 3.1 MB 00:00 2024-05-27T08:24:55.247 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:24:55.286 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 10/170 2024-05-27T08:24:55.340 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T08:24:55.375 INFO:teuthology.orchestra.run.smithi092.stdout:(150/159): librados2-19.0.0-3868.ga9cb3a58.el9. 5.7 MB/s | 3.4 MB 00:00 2024-05-27T08:24:55.552 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T08:24:55.627 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T08:24:55.647 INFO:teuthology.orchestra.run.smithi179.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 232 kB/s | 207 kB 00:00 2024-05-27T08:24:55.650 INFO:teuthology.orchestra.run.smithi092.stdout:(151/159): NetworkManager-team-1.47.90-1.el9.x8 151 kB/s | 41 kB 00:00 2024-05-27T08:24:55.654 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:24:55.742 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:24:55.764 INFO:teuthology.orchestra.run.smithi179.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 299 kB/s | 260 kB 00:00 2024-05-27T08:24:55.771 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 15/170 2024-05-27T08:24:55.817 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 15/170 2024-05-27T08:24:55.839 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T08:24:55.858 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T08:24:56.041 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T08:24:56.210 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T08:24:56.370 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T08:24:56.400 INFO:teuthology.orchestra.run.smithi092.stdout:(152/159): NetworkManager-tui-1.47.90-1.el9.x86 330 kB/s | 248 kB 00:00 2024-05-27T08:24:56.422 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T08:24:56.528 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:24:56.599 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:24:56.631 INFO:teuthology.orchestra.run.smithi179.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 1.4 MB/s | 1.3 MB 00:00 2024-05-27T08:24:56.675 INFO:teuthology.orchestra.run.smithi092.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 133 kB/s | 37 kB 00:00 2024-05-27T08:24:56.810 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:24:56.865 INFO:teuthology.orchestra.run.smithi179.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 2.4 MB 00:01 2024-05-27T08:24:56.925 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T08:24:56.953 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T08:24:56.988 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T08:24:57.062 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T08:24:57.104 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T08:24:57.157 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T08:24:57.190 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T08:24:57.225 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T08:24:57.295 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T08:24:57.334 INFO:teuthology.orchestra.run.smithi092.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 314 kB/s | 207 kB 00:00 2024-05-27T08:24:57.349 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T08:24:57.379 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T08:24:57.422 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T08:24:57.499 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T08:24:57.533 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9. 33/170 2024-05-27T08:24:57.577 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el 34/170 2024-05-27T08:24:57.605 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_6 35/170 2024-05-27T08:24:57.676 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T08:24:57.684 INFO:teuthology.orchestra.run.smithi092.stdout:(155/159): NetworkManager-1.47.90-1.el9.x86_64. 849 kB/s | 2.3 MB 00:02 2024-05-27T08:24:57.720 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T08:24:57.797 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T08:24:57.909 INFO:teuthology.orchestra.run.smithi092.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 451 kB/s | 260 kB 00:00 2024-05-27T08:24:57.930 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T08:24:57.990 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T08:24:58.118 INFO:teuthology.orchestra.run.smithi092.stdout:(157/159): NetworkManager-libnm-1.47.90-1.el9.x 635 kB/s | 1.8 MB 00:02 2024-05-27T08:24:58.520 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T08:24:58.611 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T08:24:58.639 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T08:24:58.676 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T08:24:58.717 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T08:24:58.755 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_6 46/170 2024-05-27T08:24:58.806 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T08:24:58.960 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T08:24:59.024 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T08:24:59.116 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T08:24:59.172 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T08:24:59.245 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T08:24:59.284 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T08:24:59.321 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T08:24:59.368 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T08:24:59.404 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T08:24:59.447 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T08:24:59.489 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T08:24:59.626 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T08:24:59.734 INFO:teuthology.orchestra.run.smithi092.stdout:(158/159): openssl-3.2.1-1.el9.x86_64.rpm 674 kB/s | 1.3 MB 00:02 2024-05-27T08:24:59.808 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T08:24:59.850 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T08:24:59.954 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T08:24:59.991 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T08:25:00.032 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T08:25:00.065 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T08:25:00.090 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T08:25:00.198 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T08:25:00.201 INFO:teuthology.orchestra.run.smithi092.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.0 MB/s | 2.4 MB 00:02 2024-05-27T08:25:00.205 INFO:teuthology.orchestra.run.smithi092.stdout:-------------------------------------------------------------------------------- 2024-05-27T08:25:00.205 INFO:teuthology.orchestra.run.smithi092.stdout:Total 6.1 MB/s | 227 MB 00:36 2024-05-27T08:25:00.326 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T08:25:00.364 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T08:25:00.392 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T08:25:00.427 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T08:25:00.465 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T08:25:00.506 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T08:25:00.536 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T08:25:00.570 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T08:25:00.604 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T08:25:00.637 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T08:25:00.672 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T08:25:00.759 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T08:25:00.811 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T08:25:00.841 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T08:25:00.889 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T08:25:00.987 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T08:25:01.034 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T08:25:01.415 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction check 2024-05-27T08:25:01.573 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction check succeeded. 2024-05-27T08:25:01.573 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction test 2024-05-27T08:25:01.764 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T08:25:01.808 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T08:25:01.851 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T08:25:01.952 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T08:25:02.006 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T08:25:02.051 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T08:25:02.140 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T08:25:02.170 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T08:25:02.337 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T08:25:02.368 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T08:25:02.447 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T08:25:03.108 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T08:25:03.224 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T08:25:03.448 INFO:teuthology.orchestra.run.smithi092.stdout:Transaction test succeeded. 2024-05-27T08:25:03.448 INFO:teuthology.orchestra.run.smithi092.stdout:Running transaction 2024-05-27T08:25:04.485 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T08:25:04.562 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T08:25:04.590 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T08:25:04.623 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T08:25:04.935 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T08:25:04.986 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 103/170 2024-05-27T08:25:05.060 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 103/170 2024-05-27T08:25:05.090 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 104/170 2024-05-27T08:25:05.127 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T08:25:05.606 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T08:25:05.639 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 107/170 2024-05-27T08:25:05.710 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 107/170 2024-05-27T08:25:05.740 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 108/170 2024-05-27T08:25:08.015 INFO:teuthology.orchestra.run.smithi179.stdout:(159/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 835 kB/s | 16 MB 00:19 2024-05-27T08:25:08.020 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-05-27T08:25:08.020 INFO:teuthology.orchestra.run.smithi179.stdout:Total 5.0 MB/s | 227 MB 00:45 2024-05-27T08:25:08.073 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:08.112 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:08.656 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:08.690 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 110/170 2024-05-27T08:25:08.771 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 110/170 2024-05-27T08:25:08.771 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T08:25:08.771 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T08:25:08.771 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:08.799 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 111/170 2024-05-27T08:25:09.181 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-05-27T08:25:09.339 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-05-27T08:25:09.339 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-05-27T08:25:09.689 INFO:teuthology.orchestra.run.smithi092.stdout: Preparing : 1/1 2024-05-27T08:25:09.748 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T08:25:09.787 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T08:25:09.849 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T08:25:09.883 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T08:25:09.908 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T08:25:09.951 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T08:25:10.279 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T08:25:10.401 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:25:10.521 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:25:10.576 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:25:10.793 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:25:10.822 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 10/170 2024-05-27T08:25:10.855 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T08:25:11.052 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T08:25:11.104 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T08:25:11.132 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:25:11.203 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-05-27T08:25:11.203 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-05-27T08:25:11.235 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:25:11.258 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 15/170 2024-05-27T08:25:11.321 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 15/170 2024-05-27T08:25:11.343 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T08:25:11.362 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T08:25:11.517 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T08:25:11.645 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T08:25:11.798 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T08:25:11.850 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T08:25:11.941 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:25:12.043 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:25:12.245 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:25:12.363 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T08:25:12.406 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T08:25:12.442 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T08:25:12.520 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T08:25:12.557 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T08:25:12.603 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T08:25:12.636 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T08:25:12.662 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T08:25:12.729 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T08:25:12.784 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T08:25:12.814 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T08:25:12.849 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T08:25:12.927 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T08:25:12.968 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9. 33/170 2024-05-27T08:25:13.011 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el 34/170 2024-05-27T08:25:13.040 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_6 35/170 2024-05-27T08:25:13.113 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T08:25:13.156 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T08:25:13.232 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T08:25:13.368 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T08:25:13.418 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T08:25:13.957 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T08:25:14.039 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T08:25:14.068 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T08:25:14.113 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T08:25:14.153 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T08:25:14.192 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_6 46/170 2024-05-27T08:25:14.242 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T08:25:14.394 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T08:25:14.461 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T08:25:14.555 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T08:25:14.617 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T08:25:14.690 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T08:25:14.721 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T08:25:14.759 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T08:25:14.805 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T08:25:14.840 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T08:25:14.883 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T08:25:14.926 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T08:25:15.063 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T08:25:15.102 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T08:25:15.144 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T08:25:15.239 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T08:25:15.268 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T08:25:15.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T08:25:15.334 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T08:25:15.359 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T08:25:15.435 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T08:25:15.561 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T08:25:15.621 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T08:25:15.653 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T08:25:15.687 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T08:25:15.727 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T08:25:15.758 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T08:25:15.788 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T08:25:15.823 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T08:25:15.856 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T08:25:15.889 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T08:25:15.924 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T08:25:16.011 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T08:25:16.062 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T08:25:16.094 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T08:25:16.141 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T08:25:16.240 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T08:25:16.295 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T08:25:17.040 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T08:25:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T08:25:17.117 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T08:25:17.219 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T08:25:17.265 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T08:25:17.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T08:25:17.398 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T08:25:17.419 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-05-27T08:25:17.446 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T08:25:17.487 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-05-27T08:25:17.525 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-05-27T08:25:17.587 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-05-27T08:25:17.621 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-05-27T08:25:17.622 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T08:25:17.647 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-05-27T08:25:17.651 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T08:25:17.689 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-05-27T08:25:17.728 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T08:25:18.018 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-05-27T08:25:18.123 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:25:18.231 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:25:18.257 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:25:18.357 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:25:18.387 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 10/170 2024-05-27T08:25:18.388 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T08:25:18.419 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-05-27T08:25:18.501 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T08:25:18.622 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-05-27T08:25:18.677 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-05-27T08:25:18.705 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:25:18.808 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:25:18.831 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 15/170 2024-05-27T08:25:18.893 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 15/170 2024-05-27T08:25:18.915 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T08:25:18.937 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-05-27T08:25:19.148 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T08:25:19.316 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-05-27T08:25:19.479 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T08:25:19.540 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 18/170 2024-05-27T08:25:19.634 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:25:19.708 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:25:19.740 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T08:25:19.840 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T08:25:19.868 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T08:25:19.901 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T08:25:19.902 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.90-1.el9.x86_64 19/170 2024-05-27T08:25:20.012 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-05-27T08:25:20.045 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-05-27T08:25:20.096 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/170 2024-05-27T08:25:20.161 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/170 2024-05-27T08:25:20.196 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/170 2024-05-27T08:25:20.213 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T08:25:20.239 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/170 2024-05-27T08:25:20.255 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 103/170 2024-05-27T08:25:20.272 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/170 2024-05-27T08:25:20.300 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/170 2024-05-27T08:25:20.346 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 103/170 2024-05-27T08:25:20.369 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/170 2024-05-27T08:25:20.377 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 104/170 2024-05-27T08:25:20.404 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 111/170 2024-05-27T08:25:20.404 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-05-27T08:25:20.404 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-05-27T08:25:20.404 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-05-27T08:25:20.404 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-05-27T08:25:20.404 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-05-27T08:25:20.405 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-05-27T08:25:20.405 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-05-27T08:25:20.405 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:20.414 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T08:25:20.422 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/170 2024-05-27T08:25:20.453 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/170 2024-05-27T08:25:20.488 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/170 2024-05-27T08:25:20.566 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/170 2024-05-27T08:25:20.588 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 112/170 2024-05-27T08:25:20.599 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9. 33/170 2024-05-27T08:25:20.604 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 112/170 2024-05-27T08:25:20.604 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:20.604 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T08:25:20.605 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:20.642 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el 34/170 2024-05-27T08:25:20.672 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_6 35/170 2024-05-27T08:25:20.742 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/170 2024-05-27T08:25:20.794 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/170 2024-05-27T08:25:20.869 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/170 2024-05-27T08:25:20.897 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T08:25:21.001 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/170 2024-05-27T08:25:21.055 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/170 2024-05-27T08:25:21.150 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 107/170 2024-05-27T08:25:21.221 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 107/170 2024-05-27T08:25:21.243 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 108/170 2024-05-27T08:25:21.582 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/170 2024-05-27T08:25:21.659 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/170 2024-05-27T08:25:21.705 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/170 2024-05-27T08:25:21.741 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/170 2024-05-27T08:25:21.790 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 45/170 2024-05-27T08:25:21.828 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_6 46/170 2024-05-27T08:25:21.870 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/170 2024-05-27T08:25:22.011 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 48/170 2024-05-27T08:25:22.056 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 49/170 2024-05-27T08:25:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 113/170 2024-05-27T08:25:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 113/170 2024-05-27T08:25:22.126 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:22.126 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T08:25:22.127 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T08:25:22.127 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T08:25:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:22.147 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/170 2024-05-27T08:25:22.212 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/170 2024-05-27T08:25:22.285 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-05-27T08:25:22.315 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-05-27T08:25:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 114/170 2024-05-27T08:25:22.377 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-05-27T08:25:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 114/170 2024-05-27T08:25:22.381 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:22.381 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T08:25:22.381 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T08:25:22.381 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T08:25:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:22.424 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-05-27T08:25:22.468 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-05-27T08:25:22.512 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-05-27T08:25:22.554 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-05-27T08:25:22.689 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-05-27T08:25:22.730 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-05-27T08:25:22.772 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-05-27T08:25:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 115/170 2024-05-27T08:25:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 115/170 2024-05-27T08:25:22.841 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:22.841 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T08:25:22.841 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T08:25:22.841 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T08:25:22.842 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:22.866 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-05-27T08:25:22.896 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-05-27T08:25:22.936 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-05-27T08:25:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T08:25:22.962 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-05-27T08:25:22.987 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-05-27T08:25:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T08:25:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T08:25:23.061 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-05-27T08:25:23.188 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-05-27T08:25:23.228 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-05-27T08:25:23.245 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T08:25:23.256 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-05-27T08:25:23.290 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-05-27T08:25:23.301 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T08:25:23.329 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-05-27T08:25:23.361 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-05-27T08:25:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T08:25:23.390 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-05-27T08:25:23.425 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-05-27T08:25:23.458 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-05-27T08:25:23.484 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T08:25:23.500 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-05-27T08:25:23.534 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-05-27T08:25:23.558 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:23.590 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:23.621 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-05-27T08:25:23.632 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T08:25:23.665 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-05-27T08:25:23.668 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T08:25:23.696 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-05-27T08:25:23.699 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T08:25:23.727 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T08:25:23.743 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-05-27T08:25:23.849 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-05-27T08:25:23.906 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-05-27T08:25:24.127 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:24.127 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 127/170 2024-05-27T08:25:24.159 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 127/170 2024-05-27T08:25:24.168 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 110/170 2024-05-27T08:25:24.191 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.e 128/170 2024-05-27T08:25:24.252 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58. 129/170 2024-05-27T08:25:24.262 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 110/170 2024-05-27T08:25:24.262 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T08:25:24.262 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T08:25:24.263 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:24.282 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 130/170 2024-05-27T08:25:24.286 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 111/170 2024-05-27T08:25:24.629 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-05-27T08:25:24.678 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-05-27T08:25:24.711 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-05-27T08:25:24.810 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-05-27T08:25:24.867 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-05-27T08:25:24.912 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-05-27T08:25:24.999 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-05-27T08:25:25.039 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-05-27T08:25:25.215 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-05-27T08:25:25.245 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-05-27T08:25:25.314 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 130/170 2024-05-27T08:25:25.323 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-05-27T08:25:25.355 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 131/170 2024-05-27T08:25:25.963 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-05-27T08:25:26.076 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-05-27T08:25:26.314 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 131/170 2024-05-27T08:25:26.343 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 132/170 2024-05-27T08:25:26.454 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 132/170 2024-05-27T08:25:26.614 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el 133/170 2024-05-27T08:25:26.637 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 134/170 2024-05-27T08:25:26.660 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 134/170 2024-05-27T08:25:26.660 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:26.660 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T08:25:26.660 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T08:25:26.660 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T08:25:26.661 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:26.713 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 135/170 2024-05-27T08:25:26.724 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 135/170 2024-05-27T08:25:26.821 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 136/170 2024-05-27T08:25:26.874 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 137/170 2024-05-27T08:25:27.304 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-05-27T08:25:27.390 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-05-27T08:25:27.419 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-05-27T08:25:27.452 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-05-27T08:25:27.763 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-05-27T08:25:27.801 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T08:25:27.806 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 103/170 2024-05-27T08:25:27.832 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 139/170 2024-05-27T08:25:27.855 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 139/170 2024-05-27T08:25:27.856 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:27.856 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T08:25:27.856 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T08:25:27.856 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T08:25:27.856 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:27.897 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 103/170 2024-05-27T08:25:27.899 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 140/170 2024-05-27T08:25:27.922 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 140/170 2024-05-27T08:25:27.922 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:27.922 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T08:25:27.922 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:27.928 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 104/170 2024-05-27T08:25:27.964 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-05-27T08:25:27.966 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 141/170 2024-05-27T08:25:27.990 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 141/170 2024-05-27T08:25:27.990 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:27.990 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T08:25:27.990 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T08:25:27.990 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T08:25:27.990 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:28.292 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 142/170 2024-05-27T08:25:28.315 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 142/170 2024-05-27T08:25:28.316 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:28.316 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T08:25:28.316 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T08:25:28.316 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T08:25:28.316 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:28.443 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-05-27T08:25:28.476 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 107/170 2024-05-27T08:25:28.547 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 107/170 2024-05-27T08:25:28.569 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 108/170 2024-05-27T08:25:30.789 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:30.940 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:31.481 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 109/170 2024-05-27T08:25:31.519 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 110/170 2024-05-27T08:25:31.612 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 110/170 2024-05-27T08:25:31.613 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-27T08:25:31.613 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-27T08:25:31.613 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:31.645 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 111/170 2024-05-27T08:25:32.912 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 143/170 2024-05-27T08:25:32.954 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 144/170 2024-05-27T08:25:32.994 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 145/170 2024-05-27T08:25:33.027 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T08:25:33.054 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T08:25:33.247 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 148/170 2024-05-27T08:25:33.291 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T08:25:33.323 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_ 150/170 2024-05-27T08:25:33.364 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T08:25:33.402 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T08:25:33.440 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T08:25:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:34.046 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:34.067 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:34.892 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 155/170 2024-05-27T08:25:34.983 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T08:25:35.025 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T08:25:35.106 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T08:25:35.145 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T08:25:35.170 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T08:25:35.195 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T08:25:35.222 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T08:25:35.222 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T08:25:35.306 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T08:25:35.339 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T08:25:35.339 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T08:25:35.423 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T08:25:35.448 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T08:25:35.454 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:35.454 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:35.642 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:35.642 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T08:25:35.727 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 111/170 2024-05-27T08:25:35.727 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /sys 2024-05-27T08:25:35.727 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /proc 2024-05-27T08:25:35.727 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /mnt 2024-05-27T08:25:35.727 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /var/tmp 2024-05-27T08:25:35.727 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /home 2024-05-27T08:25:35.727 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /root 2024-05-27T08:25:35.728 INFO:teuthology.orchestra.run.smithi092.stdout:skipping the directory /tmp 2024-05-27T08:25:35.728 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:35.734 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T08:25:35.759 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T08:25:35.825 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T08:25:35.918 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 112/170 2024-05-27T08:25:35.934 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 112/170 2024-05-27T08:25:35.935 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:35.935 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T08:25:35.935 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:37.396 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 113/170 2024-05-27T08:25:37.421 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 113/170 2024-05-27T08:25:37.421 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:37.422 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T08:25:37.422 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T08:25:37.422 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T08:25:37.422 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:37.646 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 114/170 2024-05-27T08:25:37.669 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 114/170 2024-05-27T08:25:37.669 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:37.669 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T08:25:37.669 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T08:25:37.669 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T08:25:37.670 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:38.114 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 115/170 2024-05-27T08:25:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 115/170 2024-05-27T08:25:38.137 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:38.137 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T08:25:38.137 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T08:25:38.137 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T08:25:38.137 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:38.252 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T08:25:38.309 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T08:25:38.365 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T08:25:38.560 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T08:25:38.606 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T08:25:38.685 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T08:25:38.775 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T08:25:38.925 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T08:25:38.973 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T08:25:39.006 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T08:25:39.033 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T08:25:39.473 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 127/170 2024-05-27T08:25:39.497 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 127/170 2024-05-27T08:25:39.538 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.e 128/170 2024-05-27T08:25:39.591 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58. 129/170 2024-05-27T08:25:39.620 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 130/170 2024-05-27T08:25:40.659 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 130/170 2024-05-27T08:25:40.697 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 131/170 2024-05-27T08:25:41.254 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T08:25:41.254 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 1/170 2024-05-27T08:25:41.254 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2/170 2024-05-27T08:25:41.254 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 3/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 4/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 5/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 6/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 7/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 10/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 11/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 12/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 13/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:25:41.255 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_ 15/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 16/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 17/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_6 18/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 19/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 20/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el 21/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9. 22/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_6 23/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 24/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 25/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 26/170 2024-05-27T08:25:41.256 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 27/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 28/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 29/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58. 30/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 31/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 32/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 33/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el 34/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 35/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.e 36/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 37/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 38/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 39/170 2024-05-27T08:25:41.257 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T08:25:41.258 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T08:25:41.259 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T08:25:41.260 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T08:25:41.261 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T08:25:41.262 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T08:25:41.263 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T08:25:41.264 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T08:25:41.265 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 149/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 151/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T08:25:41.266 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T08:25:41.267 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T08:25:41.660 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 131/170 2024-05-27T08:25:41.689 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 132/170 2024-05-27T08:25:41.801 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 132/170 2024-05-27T08:25:41.959 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el 133/170 2024-05-27T08:25:41.991 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 134/170 2024-05-27T08:25:42.014 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 134/170 2024-05-27T08:25:42.014 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:42.014 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T08:25:42.014 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T08:25:42.015 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T08:25:42.015 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:42.066 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 135/170 2024-05-27T08:25:42.077 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 135/170 2024-05-27T08:25:42.174 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 136/170 2024-05-27T08:25:42.241 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 137/170 2024-05-27T08:25:43.158 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T08:25:43.185 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 139/170 2024-05-27T08:25:43.208 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 139/170 2024-05-27T08:25:43.208 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:43.208 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T08:25:43.208 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T08:25:43.208 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T08:25:43.208 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:43.253 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 140/170 2024-05-27T08:25:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 140/170 2024-05-27T08:25:43.275 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:43.275 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T08:25:43.275 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:43.334 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 111/170 2024-05-27T08:25:43.335 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-05-27T08:25:43.335 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-05-27T08:25:43.335 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-05-27T08:25:43.335 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-05-27T08:25:43.335 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-05-27T08:25:43.335 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-05-27T08:25:43.335 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-05-27T08:25:43.335 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:43.336 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 141/170 2024-05-27T08:25:43.359 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 141/170 2024-05-27T08:25:43.359 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:43.359 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T08:25:43.359 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T08:25:43.359 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T08:25:43.359 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:43.511 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 112/170 2024-05-27T08:25:43.526 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 112/170 2024-05-27T08:25:43.526 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:43.526 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-27T08:25:43.527 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:43.671 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 142/170 2024-05-27T08:25:43.693 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 142/170 2024-05-27T08:25:43.693 INFO:teuthology.orchestra.run.smithi092.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:43.693 INFO:teuthology.orchestra.run.smithi092.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T08:25:43.693 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T08:25:43.694 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T08:25:43.694 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:44.148 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T08:25:44.148 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:44.148 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-05-27T08:25:44.148 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T08:25:44.148 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T08:25:44.148 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T08:25:44.149 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.150 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.151 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T08:25:44.152 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.153 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T08:25:44.154 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.155 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T08:25:44.156 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T08:25:44.157 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T08:25:44.158 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T08:25:44.159 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T08:25:44.160 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T08:25:44.161 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T08:25:44.161 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T08:25:44.161 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T08:25:44.161 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T08:25:44.161 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T08:25:44.161 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:25:44.161 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-05-27T08:25:44.622 DEBUG:teuthology.parallel:result is None 2024-05-27T08:25:44.989 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 113/170 2024-05-27T08:25:45.015 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 113/170 2024-05-27T08:25:45.015 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:45.015 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-27T08:25:45.015 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T08:25:45.015 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-27T08:25:45.016 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:45.240 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 114/170 2024-05-27T08:25:45.262 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 114/170 2024-05-27T08:25:45.262 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:45.262 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-27T08:25:45.262 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T08:25:45.262 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-27T08:25:45.262 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:45.699 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 115/170 2024-05-27T08:25:45.721 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 115/170 2024-05-27T08:25:45.721 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:45.721 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-27T08:25:45.721 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T08:25:45.721 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-27T08:25:45.721 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:45.829 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-05-27T08:25:45.886 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-05-27T08:25:45.966 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-05-27T08:25:46.152 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-05-27T08:25:46.200 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-05-27T08:25:46.278 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-05-27T08:25:46.365 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-05-27T08:25:46.506 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-05-27T08:25:46.558 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-05-27T08:25:46.598 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-05-27T08:25:46.626 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-05-27T08:25:47.092 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 127/170 2024-05-27T08:25:47.124 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 127/170 2024-05-27T08:25:47.157 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.e 128/170 2024-05-27T08:25:47.218 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58. 129/170 2024-05-27T08:25:47.256 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 130/170 2024-05-27T08:25:48.248 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 143/170 2024-05-27T08:25:48.290 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 144/170 2024-05-27T08:25:48.298 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 130/170 2024-05-27T08:25:48.331 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 145/170 2024-05-27T08:25:48.332 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 131/170 2024-05-27T08:25:48.363 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T08:25:48.390 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T08:25:48.578 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 148/170 2024-05-27T08:25:48.628 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T08:25:48.660 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_ 150/170 2024-05-27T08:25:48.692 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T08:25:48.722 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T08:25:48.776 INFO:teuthology.orchestra.run.smithi092.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T08:25:49.294 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 131/170 2024-05-27T08:25:49.325 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 132/170 2024-05-27T08:25:49.396 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:49.433 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:49.434 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 132/170 2024-05-27T08:25:49.451 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:49.594 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el 133/170 2024-05-27T08:25:49.619 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 134/170 2024-05-27T08:25:49.642 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 134/170 2024-05-27T08:25:49.642 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:49.642 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-27T08:25:49.642 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T08:25:49.642 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-27T08:25:49.642 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:49.686 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 135/170 2024-05-27T08:25:49.697 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 135/170 2024-05-27T08:25:49.797 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 136/170 2024-05-27T08:25:49.843 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 137/170 2024-05-27T08:25:50.284 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 155/170 2024-05-27T08:25:50.378 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T08:25:50.420 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T08:25:50.510 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T08:25:50.565 INFO:teuthology.orchestra.run.smithi092.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T08:25:50.590 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T08:25:50.615 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T08:25:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T08:25:50.642 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T08:25:50.710 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T08:25:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T08:25:50.743 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T08:25:50.768 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-05-27T08:25:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 139/170 2024-05-27T08:25:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 139/170 2024-05-27T08:25:50.819 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:50.819 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-27T08:25:50.819 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T08:25:50.819 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-27T08:25:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:50.835 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T08:25:50.860 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T08:25:50.864 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 140/170 2024-05-27T08:25:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:50.865 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:50.885 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 140/170 2024-05-27T08:25:50.886 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:50.886 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-27T08:25:50.886 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 141/170 2024-05-27T08:25:50.953 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 141/170 2024-05-27T08:25:50.953 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:50.953 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-27T08:25:50.954 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T08:25:50.954 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-27T08:25:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:51.045 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:51.045 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T08:25:51.130 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T08:25:51.155 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T08:25:51.218 INFO:teuthology.orchestra.run.smithi092.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T08:25:51.265 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 142/170 2024-05-27T08:25:51.287 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 142/170 2024-05-27T08:25:51.288 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-27T08:25:51.288 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-27T08:25:51.288 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T08:25:51.288 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-27T08:25:51.288 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:25:55.869 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 143/170 2024-05-27T08:25:55.927 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 144/170 2024-05-27T08:25:55.976 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 145/170 2024-05-27T08:25:56.016 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-05-27T08:25:56.044 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-05-27T08:25:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 148/170 2024-05-27T08:25:56.273 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-05-27T08:25:56.306 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_ 150/170 2024-05-27T08:25:56.346 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-05-27T08:25:56.376 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-team-1:1.47.90-1.el9.x86_64 152/170 2024-05-27T08:25:56.414 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-tui-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T08:25:56.944 INFO:teuthology.orchestra.run.smithi092.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 1/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 3/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 4/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 5/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 6/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 7/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 10/170 2024-05-27T08:25:56.945 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 11/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 12/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 13/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_ 15/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 16/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 17/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_6 18/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 19/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 20/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el 21/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9. 22/170 2024-05-27T08:25:56.946 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_6 23/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 24/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 25/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 26/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 27/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 28/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 29/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58. 30/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 31/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 32/170 2024-05-27T08:25:56.947 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 33/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el 34/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 35/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.e 36/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 37/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 38/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 39/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T08:25:56.948 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T08:25:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T08:25:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T08:25:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T08:25:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T08:25:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T08:25:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T08:25:56.949 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T08:25:56.950 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T08:25:56.951 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T08:25:56.952 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T08:25:56.953 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T08:25:56.954 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T08:25:56.955 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T08:25:56.956 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T08:25:56.957 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 149/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 151/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T08:25:56.958 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T08:25:56.959 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T08:25:56.972 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:57.011 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:57.031 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-05-27T08:25:57.859 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 155/170 2024-05-27T08:25:57.950 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-05-27T08:25:57.991 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-05-27T08:25:58.072 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-05-27T08:25:58.104 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-05-27T08:25:58.128 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T08:25:58.153 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-05-27T08:25:58.180 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-05-27T08:25:58.180 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T08:25:58.256 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-05-27T08:25:58.289 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-05-27T08:25:58.289 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T08:25:58.382 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-05-27T08:25:58.407 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-05-27T08:25:58.411 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:58.411 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:58.592 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-05-27T08:25:58.592 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T08:25:58.676 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-05-27T08:25:58.709 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-05-27T08:25:58.773 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout:Upgraded: 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T08:25:59.670 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout:Installed: 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.671 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.672 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.673 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.673 INFO:teuthology.orchestra.run.smithi092.stdout: cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.673 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.673 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.673 INFO:teuthology.orchestra.run.smithi092.stdout: cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:25:59.673 INFO:teuthology.orchestra.run.smithi092.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T08:25:59.674 INFO:teuthology.orchestra.run.smithi092.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T08:25:59.674 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T08:25:59.674 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T08:25:59.674 INFO:teuthology.orchestra.run.smithi092.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T08:25:59.675 INFO:teuthology.orchestra.run.smithi092.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T08:25:59.676 INFO:teuthology.orchestra.run.smithi092.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T08:25:59.677 INFO:teuthology.orchestra.run.smithi092.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T08:25:59.678 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T08:25:59.679 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T08:25:59.680 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T08:25:59.681 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T08:25:59.682 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T08:25:59.683 INFO:teuthology.orchestra.run.smithi092.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T08:25:59.684 INFO:teuthology.orchestra.run.smithi092.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T08:25:59.684 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-05-27T08:25:59.684 INFO:teuthology.orchestra.run.smithi092.stdout:Complete! 2024-05-27T08:26:00.137 DEBUG:teuthology.parallel:result is None 2024-05-27T08:26:04.541 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-05-27T08:26:04.541 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 1/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 3/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 4/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3 5/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 6/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 7/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 8/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 9/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 10/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 11/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 12/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 13/170 2024-05-27T08:26:04.542 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 14/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_ 15/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 16/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 17/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_6 18/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86 19/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 20/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el 21/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9. 22/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_6 23/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 24/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 25/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 26/170 2024-05-27T08:26:04.543 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 27/170 2024-05-27T08:26:04.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 28/170 2024-05-27T08:26:04.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 29/170 2024-05-27T08:26:04.544 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58. 30/170 2024-05-27T08:26:04.555 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noa 31/170 2024-05-27T08:26:04.555 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.n 32/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9c 33/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el 34/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 35/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.e 36/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 37/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 38/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 39/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-05-27T08:26:04.556 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-05-27T08:26:04.557 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-05-27T08:26:04.558 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-05-27T08:26:04.559 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-05-27T08:26:04.560 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-05-27T08:26:04.561 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-05-27T08:26:04.562 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-05-27T08:26:04.563 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 149/170 2024-05-27T08:26:04.564 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 151/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.47.90-1.el9.x86_64 153/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.47.90-1.el9.x86_64 155/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.47.90-1.el9.x86_64 157/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.47.90-1.el9.x86_64 159/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-05-27T08:26:04.565 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-05-27T08:26:04.566 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-05-27T08:26:04.566 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-05-27T08:26:04.566 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-05-27T08:26:04.566 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-05-27T08:26:04.566 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-05-27T08:26:07.101 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-05-27T08:26:07.101 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:26:07.101 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-05-27T08:26:07.101 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-1:1.47.90-1.el9.x86_64 2024-05-27T08:26:07.101 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm-1:1.47.90-1.el9.x86_64 2024-05-27T08:26:07.101 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team-1:1.47.90-1.el9.x86_64 2024-05-27T08:26:07.101 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui-1:1.47.90-1.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: automake-1.16.2-8.el9.noarch 2024-05-27T08:26:07.102 INFO:teuthology.orchestra.run.smithi179.stdout: bison-3.7.4-5.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.103 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.104 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top-2:19.0.0-3868.ga9cb3a58.el9.noarch 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: flex-2.6.4-9.el9.x86_64 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-27T08:26:07.105 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-27T08:26:07.106 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-27T08:26:07.107 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-27T08:26:07.107 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.107 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.107 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-27T08:26:07.107 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: lua-5.4.4-4.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks-3.9.2-5.el9.noarch 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-27T08:26:07.108 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-13.14-1.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-27T08:26:07.109 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-27T08:26:07.110 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-05-27T08:26:07.111 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-27T08:26:07.112 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-27T08:26:07.113 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-27T08:26:07.114 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-27T08:26:07.115 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:19.0.0-3868.ga9cb3a58.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-3.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-27T08:26:07.116 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-27T08:26:07.117 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-27T08:26:07.117 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:26:07.117 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-05-27T08:26:07.534 DEBUG:teuthology.parallel:result is None 2024-05-27T08:26:07.534 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:26:07.673 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T08:26:07.699 INFO:teuthology.orchestra.run.smithi077.stdout:19.0.0-3868.ga9cb3a58.el9 2024-05-27T08:26:07.700 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3868.ga9cb3a58.el9 2024-05-27T08:26:07.700 INFO:teuthology.task.install:The correct ceph version 19.0.0-3868.ga9cb3a58 is installed. 2024-05-27T08:26:07.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:26:07.839 DEBUG:teuthology.orchestra.run.smithi092:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T08:26:07.863 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-3868.ga9cb3a58.el9 2024-05-27T08:26:07.863 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3868.ga9cb3a58.el9 2024-05-27T08:26:07.863 INFO:teuthology.task.install:The correct ceph version 19.0.0-3868.ga9cb3a58 is installed. 2024-05-27T08:26:07.865 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:26:08.080 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-27T08:26:08.104 INFO:teuthology.orchestra.run.smithi179.stdout:19.0.0-3868.ga9cb3a58.el9 2024-05-27T08:26:08.104 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3868.ga9cb3a58.el9 2024-05-27T08:26:08.104 INFO:teuthology.task.install:The correct ceph version 19.0.0-3868.ga9cb3a58 is installed. 2024-05-27T08:26:08.105 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-27T08:26:08.106 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:26:08.106 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T08:26:08.138 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:26:08.138 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T08:26:08.167 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:26:08.168 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-27T08:26:08.195 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-27T08:26:08.196 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:26:08.196 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T08:26:08.225 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T08:26:08.292 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:26:08.292 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T08:26:08.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T08:26:08.385 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:26:08.385 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-05-27T08:26:08.412 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-27T08:26:08.479 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-27T08:26:08.480 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:26:08.480 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T08:26:08.510 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T08:26:08.578 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:26:08.578 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T08:26:08.601 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T08:26:08.665 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:26:08.665 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-27T08:26:08.692 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-27T08:26:08.759 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-27T08:26:08.759 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:26:08.759 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T08:26:08.784 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T08:26:08.850 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:26:08.850 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T08:26:08.877 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T08:26:08.941 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:26:08.941 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-05-27T08:26:08.968 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-27T08:26:09.035 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-27T08:26:09.125 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'a9cb3a581a309a99b72997ae5ddb88084f5484c9', 'subvols': {'create': 2}} 2024-05-27T08:26:09.125 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:26:09.126 INFO:tasks.cephadm:Cluster fsid is c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:26:09.126 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-27T08:26:09.126 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.77', 'mon.b': '172.21.15.92', 'mon.c': '172.21.15.179'} 2024-05-27T08:26:09.126 INFO:tasks.cephadm:First mon is mon.a on smithi077 2024-05-27T08:26:09.126 INFO:tasks.cephadm:First mgr is x 2024-05-27T08:26:09.127 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-27T08:26:09.127 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-05-27T08:26:09.153 DEBUG:teuthology.orchestra.run.smithi092:> sudo hostname $(hostname -s) 2024-05-27T08:26:09.177 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-05-27T08:26:09.205 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-27T08:26:09.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:26:09.426 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a9cb3a581a309a99b72997ae5ddb88084f5484c9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79711/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3868-ga9cb3a58', 'node_name': '172.21.4.69+confusa07', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3868.ga9cb3a58'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-26 17:19:17.510518', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiesce', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}] 2024-05-27T08:26:09.566 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-lusov-quiesce, sha1 a9cb3a581a309a99b72997ae5ddb88084f5484c9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:26:09.568 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/x86_64/flavors/default/cephadm 2024-05-27T08:26:09.568 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/x86_64/flavors/default/cephadm 2024-05-27T08:26:09.568 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:10.036 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 782789 May 27 08:26 /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:10.036 DEBUG:teuthology.orchestra.run.smithi092:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:10.499 INFO:teuthology.orchestra.run.smithi092.stdout:-rw-r--r--. 1 ubuntu ubuntu 782789 May 27 08:26 /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:10.500 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-lusov-quiesce/a9cb3a581a309a99b72997ae5ddb88084f5484c9/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:10.961 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 782789 May 27 08:26 /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:10.961 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:10.981 DEBUG:teuthology.orchestra.run.smithi092:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:11.000 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-27T08:26:11.101 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 on all hosts... 2024-05-27T08:26:11.101 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 pull 2024-05-27T08:26:11.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 pull 2024-05-27T08:26:11.106 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 pull 2024-05-27T08:26:11.860 INFO:teuthology.orchestra.run.smithi092.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9... 2024-05-27T08:26:11.930 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9... 2024-05-27T08:26:12.059 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9... 2024-05-27T08:26:43.140 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-27T08:26:43.141 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 19.0.0-3868-ga9cb3a58 (a9cb3a581a309a99b72997ae5ddb88084f5484c9) squid (dev)", 2024-05-27T08:26:43.141 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5", 2024-05-27T08:26:43.141 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-05-27T08:26:43.141 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7" 2024-05-27T08:26:43.141 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-05-27T08:26:43.141 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-27T08:26:43.408 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-27T08:26:43.408 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_version": "ceph version 19.0.0-3868-ga9cb3a58 (a9cb3a581a309a99b72997ae5ddb88084f5484c9) squid (dev)", 2024-05-27T08:26:43.408 INFO:teuthology.orchestra.run.smithi092.stdout: "image_id": "47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5", 2024-05-27T08:26:43.408 INFO:teuthology.orchestra.run.smithi092.stdout: "repo_digests": [ 2024-05-27T08:26:43.409 INFO:teuthology.orchestra.run.smithi092.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7" 2024-05-27T08:26:43.409 INFO:teuthology.orchestra.run.smithi092.stdout: ] 2024-05-27T08:26:43.409 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-27T08:26:46.351 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:26:46.351 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.0.0-3868-ga9cb3a58 (a9cb3a581a309a99b72997ae5ddb88084f5484c9) squid (dev)", 2024-05-27T08:26:46.351 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5", 2024-05-27T08:26:46.351 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-05-27T08:26:46.351 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7" 2024-05-27T08:26:46.351 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:26:46.351 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:26:46.375 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-05-27T08:26:46.424 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph 2024-05-27T08:26:46.457 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-05-27T08:26:46.489 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-05-27T08:26:46.515 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /etc/ceph 2024-05-27T08:26:46.543 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-05-27T08:26:46.571 INFO:tasks.cephadm:Writing seed config... 2024-05-27T08:26:46.572 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-27T08:26:46.572 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-27T08:26:46.572 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-27T08:26:46.572 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-27T08:26:46.572 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-27T08:26:46.572 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-27T08:26:46.572 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-27T08:26:46.572 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-27T08:26:46.573 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-27T08:26:46.574 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-27T08:26:46.575 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-27T08:26:46.576 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-27T08:26:46.576 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-27T08:26:46.576 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-27T08:26:46.576 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-27T08:26:46.576 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-05-27T08:26:46.577 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:26:46.577 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-27T08:26:46.592 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c4eeaaa6-1c02-11ef-bc9b-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-27T08:26:46.592 DEBUG:teuthology.orchestra.run.smithi077:mon.a> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.a.service 2024-05-27T08:26:46.635 DEBUG:teuthology.orchestra.run.smithi077:mgr.x> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.x.service 2024-05-27T08:26:46.678 INFO:tasks.cephadm:Bootstrapping... 2024-05-27T08:26:46.679 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 -v bootstrap --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.77 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:26:46.871 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-05-27T08:26:46.872 INFO:teuthology.orchestra.run.smithi077.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9', '-v', 'bootstrap', '--fsid', 'c4eeaaa6-1c02-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.77', '--skip-admin-label'] 2024-05-27T08:26:46.872 INFO:teuthology.orchestra.run.smithi077.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-27T08:26:46.872 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying podman|docker is present... 2024-05-27T08:26:46.896 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 5.0.2 2024-05-27T08:26:46.897 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying lvm2 is present... 2024-05-27T08:26:46.897 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying time synchronization is in place... 2024-05-27T08:26:46.904 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T08:26:46.904 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T08:26:46.911 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T08:26:46.911 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-05-27T08:26:46.917 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2024-05-27T08:26:46.924 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2024-05-27T08:26:46.924 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2024-05-27T08:26:46.924 INFO:teuthology.orchestra.run.smithi077.stdout:Repeating the final host check... 2024-05-27T08:26:46.947 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 5.0.2 2024-05-27T08:26:46.948 INFO:teuthology.orchestra.run.smithi077.stdout:podman (/bin/podman) version 5.0.2 is present 2024-05-27T08:26:46.948 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl is present 2024-05-27T08:26:46.948 INFO:teuthology.orchestra.run.smithi077.stdout:lvcreate is present 2024-05-27T08:26:46.954 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-27T08:26:46.954 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-27T08:26:46.962 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-27T08:26:46.962 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-05-27T08:26:46.967 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout enabled 2024-05-27T08:26:46.973 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout active 2024-05-27T08:26:46.973 INFO:teuthology.orchestra.run.smithi077.stdout:Unit chronyd.service is enabled and running 2024-05-27T08:26:46.973 INFO:teuthology.orchestra.run.smithi077.stdout:Host looks OK 2024-05-27T08:26:46.973 INFO:teuthology.orchestra.run.smithi077.stdout:Cluster fsid: c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:26:46.973 INFO:teuthology.orchestra.run.smithi077.stdout:Acquiring lock 139954886901280 on /run/cephadm/c4eeaaa6-1c02-11ef-bc9b-c7b262605968.lock 2024-05-27T08:26:46.973 INFO:teuthology.orchestra.run.smithi077.stdout:Lock 139954886901280 acquired on /run/cephadm/c4eeaaa6-1c02-11ef-bc9b-c7b262605968.lock 2024-05-27T08:26:46.974 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 3300 ... 2024-05-27T08:26:46.974 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying IP 172.21.15.77 port 6789 ... 2024-05-27T08:26:46.974 INFO:teuthology.orchestra.run.smithi077.stdout:Base mon IP(s) is [172.21.15.77:3300, 172.21.15.77:6789], mon addrv is [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-05-27T08:26:46.977 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.77 metric 100 2024-05-27T08:26:46.978 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.77 metric 100 2024-05-27T08:26:46.980 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-27T08:26:46.980 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-05-27T08:26:46.980 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1781sec hoplimit 64 pref medium 2024-05-27T08:26:46.983 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-27T08:26:46.983 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-27T08:26:46.983 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T08:26:46.983 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-05-27T08:26:46.983 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3d3/64 scope link 2024-05-27T08:26:46.983 INFO:teuthology.orchestra.run.smithi077.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-27T08:26:46.984 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2024-05-27T08:26:46.984 INFO:teuthology.orchestra.run.smithi077.stdout:Mon IP `172.21.15.77` is in CIDR network `172.21.0.0/20` 2024-05-27T08:26:46.984 INFO:teuthology.orchestra.run.smithi077.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-27T08:26:46.984 INFO:teuthology.orchestra.run.smithi077.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-27T08:26:46.985 INFO:teuthology.orchestra.run.smithi077.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9... 2024-05-27T08:26:47.461 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stdout 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 2024-05-27T08:26:47.462 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9... 2024-05-27T08:26:47.462 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Getting image source signatures 2024-05-27T08:26:47.462 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:92ea628be4077be7944dd897a7e14b19754fdad019ad944a27462acefb5053b8 2024-05-27T08:26:47.462 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying blob sha256:6ac36e7c57f227110df5cd65cfdca1cc05e0b17df6748637a6aa04c5c80f89c6 2024-05-27T08:26:47.462 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Copying config sha256:47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 2024-05-27T08:26:47.462 INFO:teuthology.orchestra.run.smithi077.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-27T08:26:48.365 INFO:teuthology.orchestra.run.smithi077.stdout:ceph: stdout ceph version 19.0.0-3868-ga9cb3a58 (a9cb3a581a309a99b72997ae5ddb88084f5484c9) squid (dev) 2024-05-27T08:26:48.366 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph version: ceph version 19.0.0-3868-ga9cb3a58 (a9cb3a581a309a99b72997ae5ddb88084f5484c9) squid (dev) 2024-05-27T08:26:48.366 INFO:teuthology.orchestra.run.smithi077.stdout:Extracting ceph user uid/gid from container image... 2024-05-27T08:26:49.405 INFO:teuthology.orchestra.run.smithi077.stdout:stat: stdout 167 167 2024-05-27T08:26:49.405 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial keys... 2024-05-27T08:26:50.244 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDJQ1RmPHgJKRAAJZWoNL+UmEi7XCtRJXhPxQ== 2024-05-27T08:26:51.091 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDKQ1Rm3rkIIxAATNDSiKeLXG75ghqxnxndBA== 2024-05-27T08:26:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph-authtool: stdout AQDLQ1RmcM3wGBAAEXcLBk564ZXA8URcCxF7YQ== 2024-05-27T08:26:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial monmap... 2024-05-27T08:26:53.037 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T08:26:53.037 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-27T08:26:53.037 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:26:53.037 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T08:26:53.037 INFO:teuthology.orchestra.run.smithi077.stdout:monmaptool for a [v2:172.21.15.77:3300,v1:172.21.15.77:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-27T08:26:53.037 INFO:teuthology.orchestra.run.smithi077.stdout:setting min_mon_release = quincy 2024-05-27T08:26:53.037 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: set fsid to c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:26:53.037 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-27T08:26:53.038 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:26:53.038 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mon... 2024-05-27T08:26:53.909 INFO:teuthology.orchestra.run.smithi077.stdout:create mon.a on 2024-05-27T08:26:54.122 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-27T08:26:54.267 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-27T08:26:54.459 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968.target. 2024-05-27T08:26:54.459 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968.target. 2024-05-27T08:26:54.695 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.a 2024-05-27T08:26:54.695 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-05-27T08:26:54.858 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968.target.wants/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@.service. 2024-05-27T08:26:55.359 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T08:26:55.359 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-05-27T08:26:55.366 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T08:26:55.367 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-05-27T08:26:55.367 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-05-27T08:26:55.367 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T08:26:55.367 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon to start... 2024-05-27T08:26:55.367 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mon... 2024-05-27T08:26:55.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:55 smithi077 systemd[1]: Started Ceph mon.a for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: monmap epoch 1 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: last_changed 2024-05-27T08:26:52.412655+0000 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: created 2024-05-27T08:26:52.412655+0000 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: min_mon_release 19 (squid) 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: election_strategy: 1 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-05-27T08:26:56.443 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: fsmap 2024-05-27T08:26:56.444 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T08:26:56.444 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: mgrmap e1: no daemons active 2024-05-27T08:26:56.444 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:56 smithi077 ceph-mon[31635]: from='client.? 172.21.15.77:0/3549593054' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-27T08:26:56.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout cluster: 2024-05-27T08:26:56.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout id: c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:26:56.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-27T08:26:56.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout services: 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.532607s) 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout data: 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout pgs: 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.795+0000 7fc3ff1d0700 1 Processor -- start 2024-05-27T08:26:56.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.795+0000 7fc3ff1d0700 1 -- start start 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.795+0000 7fc3ff1d0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f8107760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.795+0000 7fc3ff1d0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8107d30 con 0x7fc3f8107340 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.796+0000 7fc3fcf6c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f8107760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.796+0000 7fc3fcf6c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f8107760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32956/0 (socket says 172.21.15.77:32956) 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.796+0000 7fc3fcf6c700 1 -- 172.21.15.77:0/3925924634 learned_addr learned my addr 172.21.15.77:0/3925924634 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.796+0000 7fc3fcf6c700 1 -- 172.21.15.77:0/3925924634 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f810a370 con 0x7fc3f8107340 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.796+0000 7fc3fcf6c700 1 --2- 172.21.15.77:0/3925924634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f8107760 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00098a0 tx=0x7fc3f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfef08beec95a9fd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.796+0000 7fc3f77fe700 1 -- 172.21.15.77:0/3925924634 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f0004160 con 0x7fc3f8107340 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3f77fe700 1 -- 172.21.15.77:0/3925924634 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f0004300 con 0x7fc3f8107340 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3f77fe700 1 -- 172.21.15.77:0/3925924634 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f00036a0 con 0x7fc3f8107340 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3925924634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 msgr2=0x7fc3f8107760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:26:56.453 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3ff1d0700 1 --2- 172.21.15.77:0/3925924634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f8107760 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00098a0 tx=0x7fc3f0009bb0 comp rx=0 tx=0).stop 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3925924634 shutdown_connections 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3ff1d0700 1 --2- 172.21.15.77:0/3925924634 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f8107760 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3925924634 >> 172.21.15.77:0/3925924634 conn(0x7fc3f8102cb0 msgr2=0x7fc3f81050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3925924634 shutdown_connections 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.797+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3925924634 wait complete. 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3ff1d0700 1 Processor -- start 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3ff1d0700 1 -- start start 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3ff1d0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3ff1d0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f810a8a0 con 0x7fc3f8107340 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3fcf6c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3fcf6c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32966/0 (socket says 172.21.15.77:32966) 2024-05-27T08:26:56.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3fcf6c700 1 -- 172.21.15.77:0/3549593054 learned_addr learned my addr 172.21.15.77:0/3549593054 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3fcf6c700 1 -- 172.21.15.77:0/3549593054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f819cd00 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.798+0000 7fc3fcf6c700 1 --2- 172.21.15.77:0/3549593054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f819c7c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0000c00 tx=0x7fc3f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.799+0000 7fc3f5ffb700 1 -- 172.21.15.77:0/3549593054 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f0003db0 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.799+0000 7fc3f5ffb700 1 -- 172.21.15.77:0/3549593054 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f0011630 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.799+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f819cf90 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.799+0000 7fc3f5ffb700 1 -- 172.21.15.77:0/3549593054 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3f0011950 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.799+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f819fc80 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.799+0000 7fc3f5ffb700 1 -- 172.21.15.77:0/3549593054 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3f001a460 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.799+0000 7fc3f5ffb700 1 -- 172.21.15.77:0/3549593054 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc3f001b780 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.800+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f8064ba0 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.802+0000 7fc3f5ffb700 1 -- 172.21.15.77:0/3549593054 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc3f0031020 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.871+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc3f802e860 con 0x7fc3f8107340 2024-05-27T08:26:56.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.872+0000 7fc3f5ffb700 1 -- 172.21.15.77:0/3549593054 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc3f001ba20 con 0x7fc3f8107340 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.872+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 msgr2=0x7fc3f819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.872+0000 7fc3ff1d0700 1 --2- 172.21.15.77:0/3549593054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f819c7c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0000c00 tx=0x7fc3f0004750 comp rx=0 tx=0).stop 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.872+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 shutdown_connections 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.872+0000 7fc3ff1d0700 1 --2- 172.21.15.77:0/3549593054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc3f8107340 0x7fc3f819c7c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.872+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 >> 172.21.15.77:0/3549593054 conn(0x7fc3f8102cb0 msgr2=0x7fc3f81933e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.872+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 shutdown_connections 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:55.872+0000 7fc3ff1d0700 1 -- 172.21.15.77:0/3549593054 wait complete. 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:mon is available 2024-05-27T08:26:56.456 INFO:teuthology.orchestra.run.smithi077.stdout:Assimilating anything we can from ceph.conf... 2024-05-27T08:26:57.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:57 smithi077 ceph-mon[31635]: from='client.? 172.21.15.77:0/3957364747' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T08:26:57.516 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:57 smithi077 ceph-mon[31635]: from='client.? 172.21.15.77:0/3957364747' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-27T08:26:57.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:26:57.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T08:26:57.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:26:57.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T08:26:57.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.77:3300,v1:172.21.15.77:6789] 2024-05-27T08:26:57.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T08:26:57.524 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T08:26:57.525 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.869+0000 7f375086f700 1 Processor -- start 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.869+0000 7f375086f700 1 -- start start 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f375086f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f375086f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f374c108890 con 0x7f374c107f10 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f374a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f374a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32976/0 (socket says 172.21.15.77:32976) 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f374a59c700 1 -- 172.21.15.77:0/1527800846 learned_addr learned my addr 172.21.15.77:0/1527800846 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f374a59c700 1 -- 172.21.15.77:0/1527800846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c1090f0 con 0x7f374c107f10 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f374a59c700 1 --2- 172.21.15.77:0/1527800846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c108350 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f373c0098a0 tx=0x7f373c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5057bea39fec1ec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f374959a700 1 -- 172.21.15.77:0/1527800846 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f373c004160 con 0x7f374c107f10 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f374959a700 1 -- 172.21.15.77:0/1527800846 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373c004300 con 0x7f374c107f10 2024-05-27T08:26:57.526 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.870+0000 7f374959a700 1 -- 172.21.15.77:0/1527800846 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f373c0036a0 con 0x7f374c107f10 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.871+0000 7f375086f700 1 -- 172.21.15.77:0/1527800846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 msgr2=0x7f374c108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.871+0000 7f375086f700 1 --2- 172.21.15.77:0/1527800846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c108350 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f373c0098a0 tx=0x7f373c009bb0 comp rx=0 tx=0).stop 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.871+0000 7f375086f700 1 -- 172.21.15.77:0/1527800846 shutdown_connections 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.871+0000 7f375086f700 1 --2- 172.21.15.77:0/1527800846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c108350 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.871+0000 7f375086f700 1 -- 172.21.15.77:0/1527800846 >> 172.21.15.77:0/1527800846 conn(0x7f374c1034e0 msgr2=0x7f374c105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.871+0000 7f375086f700 1 -- 172.21.15.77:0/1527800846 shutdown_connections 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.871+0000 7f375086f700 1 -- 172.21.15.77:0/1527800846 wait complete. 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.872+0000 7f375086f700 1 Processor -- start 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.872+0000 7f375086f700 1 -- start start 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.872+0000 7f375086f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.872+0000 7f375086f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1094a0 con 0x7f374c107f10 2024-05-27T08:26:57.527 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.872+0000 7f374a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.872+0000 7f374a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c19c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32984/0 (socket says 172.21.15.77:32984) 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.872+0000 7f374a59c700 1 -- 172.21.15.77:0/3957364747 learned_addr learned my addr 172.21.15.77:0/3957364747 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.872+0000 7f374a59c700 1 -- 172.21.15.77:0/3957364747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c19cd40 con 0x7f374c107f10 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.873+0000 7f374a59c700 1 --2- 172.21.15.77:0/3957364747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c19c800 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f373c000c00 tx=0x7f373c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.873+0000 7f373b7fe700 1 -- 172.21.15.77:0/3957364747 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f373c003db0 con 0x7f374c107f10 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.873+0000 7f373b7fe700 1 -- 172.21.15.77:0/3957364747 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373c011630 con 0x7f374c107f10 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.873+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c19cfd0 con 0x7f374c107f10 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.873+0000 7f373b7fe700 1 -- 172.21.15.77:0/3957364747 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f373c011950 con 0x7f374c107f10 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.873+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c19fcc0 con 0x7f374c107f10 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.873+0000 7f373b7fe700 1 -- 172.21.15.77:0/3957364747 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f373c01a460 con 0x7f374c107f10 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.873+0000 7f373b7fe700 1 -- 172.21.15.77:0/3957364747 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f373c01b780 con 0x7f374c107f10 2024-05-27T08:26:57.528 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.874+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374c064ba0 con 0x7f374c107f10 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.876+0000 7f373b7fe700 1 -- 172.21.15.77:0/3957364747 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f373c031020 con 0x7f374c107f10 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.943+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f374c02e860 con 0x7f374c107f10 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.946+0000 7f373b7fe700 1 -- 172.21.15.77:0/3957364747 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f373c01b980 con 0x7f374c107f10 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.947+0000 7f373b7fe700 1 -- 172.21.15.77:0/3957364747 <== mon.0 v2:172.21.15.77:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f373c01a840 con 0x7f374c107f10 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.947+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 msgr2=0x7f374c19c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.947+0000 7f375086f700 1 --2- 172.21.15.77:0/3957364747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c19c800 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f373c000c00 tx=0x7f373c004750 comp rx=0 tx=0).stop 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.947+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 shutdown_connections 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.947+0000 7f375086f700 1 --2- 172.21.15.77:0/3957364747 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f374c107f10 0x7f374c19c800 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.947+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 >> 172.21.15.77:0/3957364747 conn(0x7f374c1034e0 msgr2=0x7f374c18f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.947+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 shutdown_connections 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:56.947+0000 7f375086f700 1 -- 172.21.15.77:0/3957364747 wait complete. 2024-05-27T08:26:57.529 INFO:teuthology.orchestra.run.smithi077.stdout:Generating new minimal ceph.conf... 2024-05-27T08:26:58.622 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:58 smithi077 ceph-mon[31635]: from='client.? 172.21.15.77:0/2917219560' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:26:58.628 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.958+0000 7eff3cb7f700 1 Processor -- start 2024-05-27T08:26:58.628 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.958+0000 7eff3cb7f700 1 -- start start 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3cb7f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38108350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3cb7f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff38108890 con 0x7eff38107f10 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32986/0 (socket says 172.21.15.77:32986) 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3659c700 1 -- 172.21.15.77:0/3014565686 learned_addr learned my addr 172.21.15.77:0/3014565686 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3659c700 1 -- 172.21.15.77:0/3014565686 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff381090f0 con 0x7eff38107f10 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3659c700 1 --2- 172.21.15.77:0/3014565686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38108350 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7eff280098a0 tx=0x7eff28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2466da5927a2665 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3559a700 1 -- 172.21.15.77:0/3014565686 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff28004160 con 0x7eff38107f10 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3559a700 1 -- 172.21.15.77:0/3014565686 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff28004300 con 0x7eff38107f10 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.959+0000 7eff3559a700 1 -- 172.21.15.77:0/3014565686 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff28011af0 con 0x7eff38107f10 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.960+0000 7eff3cb7f700 1 -- 172.21.15.77:0/3014565686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 msgr2=0x7eff38108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:26:58.629 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.960+0000 7eff3cb7f700 1 --2- 172.21.15.77:0/3014565686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38108350 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7eff280098a0 tx=0x7eff28009bb0 comp rx=0 tx=0).stop 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.960+0000 7eff3cb7f700 1 -- 172.21.15.77:0/3014565686 shutdown_connections 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.960+0000 7eff3cb7f700 1 --2- 172.21.15.77:0/3014565686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38108350 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.960+0000 7eff3cb7f700 1 -- 172.21.15.77:0/3014565686 >> 172.21.15.77:0/3014565686 conn(0x7eff381034e0 msgr2=0x7eff38105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.960+0000 7eff3cb7f700 1 -- 172.21.15.77:0/3014565686 shutdown_connections 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.960+0000 7eff3cb7f700 1 -- 172.21.15.77:0/3014565686 wait complete. 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3cb7f700 1 Processor -- start 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3cb7f700 1 -- start start 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3cb7f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38076a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3cb7f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7eff38109f80 con 0x7eff38107f10 2024-05-27T08:26:58.630 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38076a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38076a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:32994/0 (socket says 172.21.15.77:32994) 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3659c700 1 -- 172.21.15.77:0/2917219560 learned_addr learned my addr 172.21.15.77:0/2917219560 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3659c700 1 -- 172.21.15.77:0/2917219560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3819fcf0 con 0x7eff38107f10 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.961+0000 7eff3659c700 1 --2- 172.21.15.77:0/2917219560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38076a40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7eff280117b0 tx=0x7eff28011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.962+0000 7eff277fe700 1 -- 172.21.15.77:0/2917219560 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff28003af0 con 0x7eff38107f10 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.962+0000 7eff277fe700 1 -- 172.21.15.77:0/2917219560 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff28020070 con 0x7eff38107f10 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.962+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3819fec0 con 0x7eff38107f10 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.962+0000 7eff277fe700 1 -- 172.21.15.77:0/2917219560 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff28019630 con 0x7eff38107f10 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.962+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff381a02e0 con 0x7eff38107f10 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.962+0000 7eff277fe700 1 -- 172.21.15.77:0/2917219560 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff28019c80 con 0x7eff38107f10 2024-05-27T08:26:58.631 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.962+0000 7eff277fe700 1 -- 172.21.15.77:0/2917219560 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7eff2800b720 con 0x7eff38107f10 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.963+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff38064ba0 con 0x7eff38107f10 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:57.965+0000 7eff277fe700 1 -- 172.21.15.77:0/2917219560 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7eff280042a0 con 0x7eff38107f10 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.031+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7eff381a0a80 con 0x7eff38107f10 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.031+0000 7eff277fe700 1 -- 172.21.15.77:0/2917219560 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7eff2801a9c0 con 0x7eff38107f10 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.032+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 msgr2=0x7eff38076a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.032+0000 7eff3cb7f700 1 --2- 172.21.15.77:0/2917219560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38076a40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7eff280117b0 tx=0x7eff28011b70 comp rx=0 tx=0).stop 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.032+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 shutdown_connections 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.032+0000 7eff3cb7f700 1 --2- 172.21.15.77:0/2917219560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7eff38107f10 0x7eff38076a40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.032+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 >> 172.21.15.77:0/2917219560 conn(0x7eff381034e0 msgr2=0x7eff381040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.032+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 shutdown_connections 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:26:58.032+0000 7eff3cb7f700 1 -- 172.21.15.77:0/2917219560 wait complete. 2024-05-27T08:26:58.632 INFO:teuthology.orchestra.run.smithi077.stdout:Restarting the monitor... 2024-05-27T08:26:58.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:58 smithi077 systemd[1]: Stopping Ceph mon.a for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:26:59.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:58 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a[31631]: 2024-05-27T08:26:58.907+0000 7faa7776c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:26:59.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:58 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a[31631]: 2024-05-27T08:26:58.907+0000 7faa7776c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-27T08:26:59.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:59 smithi077 podman[31882]: 2024-05-27 08:26:59.123117797 +0000 UTC m=+0.324589885 container died 49ab5ba11e2a51cb814e9ae5cfe033f227f295cc7862f7e3fb1cdf68cd138160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:26:59.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:59 smithi077 podman[31882]: 2024-05-27 08:26:59.424208595 +0000 UTC m=+0.625680683 container cleanup 49ab5ba11e2a51cb814e9ae5cfe033f227f295cc7862f7e3fb1cdf68cd138160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:26:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:59 smithi077 bash[31882]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a 2024-05-27T08:27:00.109 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:59 smithi077 podman[31894]: 2024-05-27 08:26:59.650172909 +0000 UTC m=+0.531647210 container remove 49ab5ba11e2a51cb814e9ae5cfe033f227f295cc7862f7e3fb1cdf68cd138160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-27T08:27:00.109 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:59 smithi077 systemd[1]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-27T08:27:00.110 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:59 smithi077 systemd[1]: Stopped Ceph mon.a for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:27:00.110 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:26:59 smithi077 systemd[1]: Starting Ceph mon.a for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 podman[31988]: 2024-05-27 08:27:00.011716083 +0000 UTC m=+0.016953397 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 podman[31988]: 2024-05-27 08:27:00.118069849 +0000 UTC m=+0.123307149 container create 59b893a782958c7fffb0ecc822012fdda7d5555885633545af796e8402489e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=) 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 podman[31988]: 2024-05-27 08:27:00.328566757 +0000 UTC m=+0.333804057 container init 59b893a782958c7fffb0ecc822012fdda7d5555885633545af796e8402489e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2) 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 podman[31988]: 2024-05-27 08:27:00.331709579 +0000 UTC m=+0.336946880 container start 59b893a782958c7fffb0ecc822012fdda7d5555885633545af796e8402489e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: set uid:gid to 167:167 (ceph:ceph) 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: ceph version 19.0.0-3868-ga9cb3a58 (a9cb3a581a309a99b72997ae5ddb88084f5484c9) squid (dev), process ceph-mon, pid 2 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: pidfile_write: ignore empty --pid-file 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: load: jerasure load: lrc 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: RocksDB version: 7.9.2 2024-05-27T08:27:00.360 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Git sha 0 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Compile date 2024-05-26 14:32:23 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: DB SUMMARY 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: DB Session ID: TNIJ8L9OYVRRGXJTHQ9K 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: CURRENT file: CURRENT 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: IDENTITY file: IDENTITY 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105442 ; 2024-05-27T08:27:00.361 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.error_if_exists: 0 2024-05-27T08:27:00.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.create_if_missing: 0 2024-05-27T08:27:00.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.paranoid_checks: 1 2024-05-27T08:27:00.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-27T08:27:00.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-27T08:27:00.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-27T08:27:00.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.env: 0x563d70644cc0 2024-05-27T08:27:00.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.fs: PosixFileSystem 2024-05-27T08:27:00.362 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.info_log: 0x563d711d3da0 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_file_opening_threads: 16 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.statistics: (nil) 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.use_fsync: 0 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_log_file_size: 0 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.keep_log_file_num: 1000 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.recycle_log_file_num: 0 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.allow_fallocate: 1 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.allow_mmap_reads: 0 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.allow_mmap_writes: 0 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.use_direct_reads: 0 2024-05-27T08:27:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.create_missing_column_families: 0 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.db_log_dir: 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.wal_dir: 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.advise_random_on_open: 1 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.db_write_buffer_size: 0 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.write_buffer_manager: 0x563d711a50e0 2024-05-27T08:27:00.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.rate_limiter: (nil) 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.wal_recovery_mode: 2 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.enable_thread_tracking: 0 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.enable_pipelined_write: 0 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.unordered_write: 0 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-27T08:27:00.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.row_cache: None 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.wal_filter: None 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.allow_ingest_behind: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.two_write_queues: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.manual_wal_flush: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.wal_compression: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.atomic_flush: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.log_readahead_size: 0 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-27T08:27:00.366 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.best_efforts_recovery: 0 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.allow_data_in_errors: 0 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.db_host_id: __hostname__ 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_background_jobs: 2 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_background_compactions: -1 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_subcompactions: 1 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_total_wal_size: 0 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-27T08:27:00.367 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_open_files: -1 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bytes_per_sync: 0 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_readahead_size: 0 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_background_flushes: -1 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Compression algorithms supported: 2024-05-27T08:27:00.368 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: kZSTD supported: 0 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: kXpressCompression supported: 0 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: kLZ4HCCompression supported: 1 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: kZlibCompression supported: 1 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: kSnappyCompression supported: 1 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: kLZ4Compression supported: 1 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: kBZip2Compression supported: 0 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-27T08:27:00.369 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.merge_operator: 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_filter: None 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_filter_factory: None 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.sst_partitioner_factory: None 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563d711d3f60) 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout: cache_index_and_filter_blocks: 1 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-27T08:27:00.370 INFO:journalctl@ceph.mon.a.smithi077.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: pin_top_level_index_and_filter: 1 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_type: 0 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: data_block_index_type: 0 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_shortening: 1 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: checksum: 4 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: no_block_cache: 0 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache: 0x563d711d71f0 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_name: BinnedLRUCache 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_options: 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: capacity : 536870912 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: num_shard_bits : 4 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: strict_capacity_limit : 0 2024-05-27T08:27:00.371 INFO:journalctl@ceph.mon.a.smithi077.stdout: high_pri_pool_ratio: 0.000 2024-05-27T08:27:00.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_cache_compressed: (nil) 2024-05-27T08:27:00.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: persistent_cache: (nil) 2024-05-27T08:27:00.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_size: 4096 2024-05-27T08:27:00.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_size_deviation: 10 2024-05-27T08:27:00.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_restart_interval: 16 2024-05-27T08:27:00.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: index_block_restart_interval: 1 2024-05-27T08:27:00.372 INFO:journalctl@ceph.mon.a.smithi077.stdout: metadata_block_size: 4096 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: partition_filters: 0 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: use_delta_encoding: 1 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: filter_policy: bloomfilter 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: whole_key_filtering: 1 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: verify_compression: 0 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: read_amp_bytes_per_bit: 0 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: format_version: 5 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: enable_index_compression: 1 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: block_align: 0 2024-05-27T08:27:00.373 INFO:journalctl@ceph.mon.a.smithi077.stdout: max_auto_readahead_size: 262144 2024-05-27T08:27:00.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: prepopulate_block_cache: 0 2024-05-27T08:27:00.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: initial_auto_readahead_size: 8192 2024-05-27T08:27:00.374 INFO:journalctl@ceph.mon.a.smithi077.stdout: num_file_reads_for_auto_readahead: 2 2024-05-27T08:27:00.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.write_buffer_size: 33554432 2024-05-27T08:27:00.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_write_buffer_number: 2 2024-05-27T08:27:00.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression: NoCompression 2024-05-27T08:27:00.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression: Disabled 2024-05-27T08:27:00.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.prefix_extractor: nullptr 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.num_levels: 7 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-27T08:27:00.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-27T08:27:00.408 INFO:teuthology.orchestra.run.smithi077.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-27T08:27:00.624 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-27T08:27:00.624 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-27T08:27:00.624 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-27T08:27:00.624 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.level: 32767 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.strategy: 0 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.enabled: false 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.target_file_size_base: 67108864 2024-05-27T08:27:00.625 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-27T08:27:00.626 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.arena_block_size: 1048576 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.disable_auto_compactions: 0 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-27T08:27:00.627 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.inplace_update_support: 0 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.bloom_locality: 0 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.max_successive_merges: 0 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.paranoid_file_checks: 0 2024-05-27T08:27:00.628 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.force_consistency_checks: 1 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.report_bg_io_stats: 0 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.ttl: 2592000 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.enable_blob_files: false 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.min_blob_size: 0 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.blob_file_size: 268435456 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-27T08:27:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.blob_file_starting_level: 0 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 49c142c1-1490-4dcc-9735-e89ab0a4421b 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716798420360424, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716798420361459, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98439, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14264, "raw_average_key_size": 49, "raw_value_size": 90693, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1716798420, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "49c142c1-1490-4dcc-9735-e89ab0a4421b", "db_session_id": "TNIJ8L9OYVRRGXJTHQ9K", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: EVENT_LOG_v1 {"time_micros": 1716798420361539, "job": 1, "event": "recovery_finished"} 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563d712f4000 2024-05-27T08:27:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: DB pointer 0x563d712e0000 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** DB Stats ** 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** Compaction Stats [default] ** 2024-05-27T08:27:00.631 INFO:journalctl@ceph.mon.a.smithi077.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: L0 2/0 100.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: Sum 2/0 100.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 161.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** Compaction Stats [default] ** 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 161.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-05-27T08:27:00.632 INFO:journalctl@ceph.mon.a.smithi077.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: Cumulative compaction: 0.00 GB write, 27.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: Interval compaction: 0.00 GB write, 27.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: Block cache BinnedLRUCache@0x563d711d71f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: starting mon.a rank 0 at public addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] at bind addrs [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:27:00.633 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a@-1(???) e1 preinit fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a@-1(???).mds e1 new map 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a@-1(???).mds e1 print_map 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout: e1 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout: legacy client fscid: -1 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout: 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout: No filesystems configured 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T08:27:00.634 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 bash[31988]: 59b893a782958c7fffb0ecc822012fdda7d5555885633545af796e8402489e34 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 systemd[1]: Started Ceph mon.a for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: monmap epoch 1 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: last_changed 2024-05-27T08:26:52.412655+0000 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: created 2024-05-27T08:26:52.412655+0000 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: min_mon_release 19 (squid) 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: election_strategy: 1 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: fsmap 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: osdmap e1: 0 total, 0 up, 0 in 2024-05-27T08:27:00.635 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:00 smithi077 ceph-mon[32007]: mgrmap e1: no daemons active 2024-05-27T08:27:01.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.809+0000 7f7d411c1700 1 Processor -- start 2024-05-27T08:27:01.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.809+0000 7f7d411c1700 1 -- start start 2024-05-27T08:27:01.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.809+0000 7f7d411c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c107750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.809+0000 7f7d411c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c107d20 con 0x7f7d3c107330 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.809+0000 7f7d3ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c107750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.809+0000 7f7d3ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c107750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52472/0 (socket says 172.21.15.77:52472) 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.809+0000 7f7d3ad9d700 1 -- 172.21.15.77:0/1293953404 learned_addr learned my addr 172.21.15.77:0/1293953404 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d3ad9d700 1 -- 172.21.15.77:0/1293953404 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c10a360 con 0x7f7d3c107330 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d3ad9d700 1 --2- 172.21.15.77:0/1293953404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c107750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0098a0 tx=0x7f7d2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dfbbe42dd5ac80 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d39d9b700 1 -- 172.21.15.77:0/1293953404 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d2c004160 con 0x7f7d3c107330 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d39d9b700 1 -- 172.21.15.77:0/1293953404 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7d2c004300 con 0x7f7d3c107330 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d39d9b700 1 -- 172.21.15.77:0/1293953404 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d2c011af0 con 0x7f7d3c107330 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d411c1700 1 -- 172.21.15.77:0/1293953404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 msgr2=0x7f7d3c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d411c1700 1 --2- 172.21.15.77:0/1293953404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c107750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0098a0 tx=0x7f7d2c009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:01.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d411c1700 1 -- 172.21.15.77:0/1293953404 shutdown_connections 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d411c1700 1 --2- 172.21.15.77:0/1293953404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c107750 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d411c1700 1 -- 172.21.15.77:0/1293953404 >> 172.21.15.77:0/1293953404 conn(0x7f7d3c102d60 msgr2=0x7f7d3c105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d411c1700 1 -- 172.21.15.77:0/1293953404 shutdown_connections 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.810+0000 7f7d411c1700 1 -- 172.21.15.77:0/1293953404 wait complete. 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.811+0000 7f7d411c1700 1 Processor -- start 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.811+0000 7f7d411c1700 1 -- start start 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d411c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d411c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c10d590 con 0x7f7d3c107330 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d3ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d3ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52482/0 (socket says 172.21.15.77:52482) 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d3ad9d700 1 -- 172.21.15.77:0/694964452 learned_addr learned my addr 172.21.15.77:0/694964452 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d3ad9d700 1 -- 172.21.15.77:0/694964452 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c10a120 con 0x7f7d3c107330 2024-05-27T08:27:01.460 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d3ad9d700 1 --2- 172.21.15.77:0/694964452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c109be0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009fb0 tx=0x7f7d2c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d23fff700 1 -- 172.21.15.77:0/694964452 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d2c0036a0 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c108310 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c108850 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d23fff700 1 -- 172.21.15.77:0/694964452 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7d2c004300 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.812+0000 7f7d23fff700 1 -- 172.21.15.77:0/694964452 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d2c01ab50 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.813+0000 7f7d23fff700 1 -- 172.21.15.77:0/694964452 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d2c02d7d0 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.813+0000 7f7d23fff700 1 -- 172.21.15.77:0/694964452 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7d2c019700 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.813+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d28005360 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.816+0000 7f7d23fff700 1 -- 172.21.15.77:0/694964452 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7d2c01a430 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.883+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f7d28005650 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.884+0000 7f7d23fff700 1 -- 172.21.15.77:0/694964452 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f7d2c032030 con 0x7f7d3c107330 2024-05-27T08:27:01.461 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.885+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 msgr2=0x7f7d3c109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:01.462 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.885+0000 7f7d411c1700 1 --2- 172.21.15.77:0/694964452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c109be0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009fb0 tx=0x7f7d2c011940 comp rx=0 tx=0).stop 2024-05-27T08:27:01.462 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.885+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 shutdown_connections 2024-05-27T08:27:01.462 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.885+0000 7f7d411c1700 1 --2- 172.21.15.77:0/694964452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d3c107330 0x7f7d3c109be0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:01.462 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.885+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 >> 172.21.15.77:0/694964452 conn(0x7f7d3c102d60 msgr2=0x7f7d3c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:01.462 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.885+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 shutdown_connections 2024-05-27T08:27:01.462 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:00.885+0000 7f7d411c1700 1 -- 172.21.15.77:0/694964452 wait complete. 2024-05-27T08:27:01.462 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-27T08:27:01.485 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:27:01.485 INFO:teuthology.orchestra.run.smithi077.stdout:Creating mgr... 2024-05-27T08:27:01.485 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-27T08:27:01.486 INFO:teuthology.orchestra.run.smithi077.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-27T08:27:01.737 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T08:27:01.973 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-05-27T08:27:01.974 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968.target.wants/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@.service. 2024-05-27T08:27:02.235 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:01 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/694964452' entity='client.admin' 2024-05-27T08:27:02.406 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T08:27:02.406 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-05-27T08:27:02.414 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T08:27:02.414 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-05-27T08:27:02.414 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-05-27T08:27:02.415 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to enable service . firewalld.service is not available 2024-05-27T08:27:02.425 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T08:27:02.425 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-05-27T08:27:02.434 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T08:27:02.434 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-05-27T08:27:02.434 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-05-27T08:27:02.434 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-27T08:27:02.434 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr to start... 2024-05-27T08:27:02.434 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr... 2024-05-27T08:27:03.333 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:03 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1450338681' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T08:27:03.334 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:03 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:03.195+0000 7f28c4926200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T08:27:03.550 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:27:03.550 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T08:27:03.551 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:03.552 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T08:27:03.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T08:27:03.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T08:27:03.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T08:27:03.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T08:26:55.342377+0000", 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-05-27T08:27:03.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.910+0000 7fadceaf3700 1 Processor -- start 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.910+0000 7fadceaf3700 1 -- start start 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.910+0000 7fadceaf3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc810bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.910+0000 7fadceaf3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8076e20 con 0x7fadc8109af0 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.911+0000 7fadcc88f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc810bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.911+0000 7fadcc88f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc810bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52504/0 (socket says 172.21.15.77:52504) 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.911+0000 7fadcc88f700 1 -- 172.21.15.77:0/3224430165 learned_addr learned my addr 172.21.15.77:0/3224430165 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.911+0000 7fadcc88f700 1 -- 172.21.15.77:0/3224430165 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc810c420 con 0x7fadc8109af0 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.911+0000 7fadcc88f700 1 --2- 172.21.15.77:0/3224430165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc810bee0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098a0 tx=0x7fadbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1ab76041902352 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.911+0000 7fadc77fe700 1 -- 172.21.15.77:0/3224430165 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadbc004160 con 0x7fadc8109af0 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.911+0000 7fadc77fe700 1 -- 172.21.15.77:0/3224430165 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fadbc004300 con 0x7fadc8109af0 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.911+0000 7fadc77fe700 1 -- 172.21.15.77:0/3224430165 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadbc011af0 con 0x7fadc8109af0 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.912+0000 7fadceaf3700 1 -- 172.21.15.77:0/3224430165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 msgr2=0x7fadc810bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:03.563 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.912+0000 7fadceaf3700 1 --2- 172.21.15.77:0/3224430165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc810bee0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0098a0 tx=0x7fadbc009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.912+0000 7fadceaf3700 1 -- 172.21.15.77:0/3224430165 shutdown_connections 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.912+0000 7fadceaf3700 1 --2- 172.21.15.77:0/3224430165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc810bee0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.912+0000 7fadceaf3700 1 -- 172.21.15.77:0/3224430165 >> 172.21.15.77:0/3224430165 conn(0x7fadc81034e0 msgr2=0x7fadc8105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.912+0000 7fadceaf3700 1 -- 172.21.15.77:0/3224430165 shutdown_connections 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.912+0000 7fadceaf3700 1 -- 172.21.15.77:0/3224430165 wait complete. 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadceaf3700 1 Processor -- start 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadceaf3700 1 -- start start 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadceaf3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadceaf3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadc810d370 con 0x7fadc8109af0 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadcc88f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadcc88f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52506/0 (socket says 172.21.15.77:52506) 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadcc88f700 1 -- 172.21.15.77:0/1450338681 learned_addr learned my addr 172.21.15.77:0/1450338681 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:03.564 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadcc88f700 1 -- 172.21.15.77:0/1450338681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc819d500 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadcc88f700 1 --2- 172.21.15.77:0/1450338681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc819cfc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009fb0 tx=0x7fadbc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadc5ffb700 1 -- 172.21.15.77:0/1450338681 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadbc0036a0 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadc5ffb700 1 -- 172.21.15.77:0/1450338681 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fadbc01a430 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc819d7f0 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.913+0000 7fadc5ffb700 1 -- 172.21.15.77:0/1450338681 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadbc019550 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.914+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc81a0480 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.914+0000 7fadc5ffb700 1 -- 172.21.15.77:0/1450338681 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fadbc019be0 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.914+0000 7fadc5ffb700 1 -- 172.21.15.77:0/1450338681 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fadbc024730 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.915+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc8064ba0 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.917+0000 7fadc5ffb700 1 -- 172.21.15.77:0/1450338681 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fadbc01a5e0 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.989+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fadc81a0aa0 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.989+0000 7fadc5ffb700 1 -- 172.21.15.77:0/1450338681 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fadbc01a8e0 con 0x7fadc8109af0 2024-05-27T08:27:03.565 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.991+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 msgr2=0x7fadc819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:03.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.991+0000 7fadceaf3700 1 --2- 172.21.15.77:0/1450338681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc819cfc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009fb0 tx=0x7fadbc011940 comp rx=0 tx=0).stop 2024-05-27T08:27:03.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.991+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 shutdown_connections 2024-05-27T08:27:03.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.991+0000 7fadceaf3700 1 --2- 172.21.15.77:0/1450338681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadc8109af0 0x7fadc819cfc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:03.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.991+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 >> 172.21.15.77:0/1450338681 conn(0x7fadc81034e0 msgr2=0x7fadc8104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:03.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.991+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 shutdown_connections 2024-05-27T08:27:03.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:02.991+0000 7fadceaf3700 1 -- 172.21.15.77:0/1450338681 wait complete. 2024-05-27T08:27:03.566 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (1/15)... 2024-05-27T08:27:04.629 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:04 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:04.371+0000 7f28c4926200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T08:27:04.629 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:04 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:04.536+0000 7f28c4926200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T08:27:04.986 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:04 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:04.704+0000 7f28c4926200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T08:27:04.987 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:04 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:04.985+0000 7f28c4926200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T08:27:05.262 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:05 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:05.075+0000 7f28c4926200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T08:27:05.629 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:05 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:05.281+0000 7f28c4926200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T08:27:05.629 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:05 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:05.473+0000 7f28c4926200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:27:06.319 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:06 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1645965070' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T08:27:06.320 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:06 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:06.097+0000 7f28c4926200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T08:27:06.320 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:06 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:06.318+0000 7f28c4926200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:06.705 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T08:27:06.706 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T08:27:06.707 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T08:27:06.708 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T08:26:55.342377+0000", 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:06.709 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.938+0000 7fde4473f700 1 Processor -- start 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.938+0000 7fde4473f700 1 -- start start 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.938+0000 7fde4473f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c10bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.938+0000 7fde4473f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c07cde0 con 0x7fde3c109b70 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.938+0000 7fde424db700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c10bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.938+0000 7fde424db700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c10bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52512/0 (socket says 172.21.15.77:52512) 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.938+0000 7fde424db700 1 -- 172.21.15.77:0/3621501205 learned_addr learned my addr 172.21.15.77:0/3621501205 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.939+0000 7fde424db700 1 -- 172.21.15.77:0/3621501205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c10c4a0 con 0x7fde3c109b70 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.939+0000 7fde424db700 1 --2- 172.21.15.77:0/3621501205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c10bf60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde380098a0 tx=0x7fde38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7bf44233d46baf server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.939+0000 7fde414d9700 1 -- 172.21.15.77:0/3621501205 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde38004160 con 0x7fde3c109b70 2024-05-27T08:27:06.710 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.939+0000 7fde414d9700 1 -- 172.21.15.77:0/3621501205 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde38004300 con 0x7fde3c109b70 2024-05-27T08:27:06.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.939+0000 7fde414d9700 1 -- 172.21.15.77:0/3621501205 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde38011af0 con 0x7fde3c109b70 2024-05-27T08:27:06.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.940+0000 7fde4473f700 1 -- 172.21.15.77:0/3621501205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 msgr2=0x7fde3c10bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:06.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.940+0000 7fde4473f700 1 --2- 172.21.15.77:0/3621501205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c10bf60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde380098a0 tx=0x7fde38009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:06.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.940+0000 7fde4473f700 1 -- 172.21.15.77:0/3621501205 shutdown_connections 2024-05-27T08:27:06.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.940+0000 7fde4473f700 1 --2- 172.21.15.77:0/3621501205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c10bf60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:06.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.940+0000 7fde4473f700 1 -- 172.21.15.77:0/3621501205 >> 172.21.15.77:0/3621501205 conn(0x7fde3c1034e0 msgr2=0x7fde3c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:06.711 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.940+0000 7fde4473f700 1 -- 172.21.15.77:0/3621501205 shutdown_connections 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.940+0000 7fde4473f700 1 -- 172.21.15.77:0/3621501205 wait complete. 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde4473f700 1 Processor -- start 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde4473f700 1 -- start start 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde4473f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde4473f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c10d3b0 con 0x7fde3c109b70 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde424db700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde424db700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52514/0 (socket says 172.21.15.77:52514) 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde424db700 1 -- 172.21.15.77:0/1645965070 learned_addr learned my addr 172.21.15.77:0/1645965070 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde424db700 1 -- 172.21.15.77:0/1645965070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c19d4f0 con 0x7fde3c109b70 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde424db700 1 --2- 172.21.15.77:0/1645965070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c19cfb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde38009fb0 tx=0x7fde38011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde337fe700 1 -- 172.21.15.77:0/1645965070 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde380036a0 con 0x7fde3c109b70 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde337fe700 1 -- 172.21.15.77:0/1645965070 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde3801a430 con 0x7fde3c109b70 2024-05-27T08:27:06.712 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c19d780 con 0x7fde3c109b70 2024-05-27T08:27:06.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.941+0000 7fde337fe700 1 -- 172.21.15.77:0/1645965070 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde38019550 con 0x7fde3c109b70 2024-05-27T08:27:06.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.942+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c1a0470 con 0x7fde3c109b70 2024-05-27T08:27:06.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.942+0000 7fde337fe700 1 -- 172.21.15.77:0/1645965070 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde38019be0 con 0x7fde3c109b70 2024-05-27T08:27:06.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.942+0000 7fde337fe700 1 -- 172.21.15.77:0/1645965070 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde38024730 con 0x7fde3c109b70 2024-05-27T08:27:06.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.942+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3c064ba0 con 0x7fde3c109b70 2024-05-27T08:27:06.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:05.945+0000 7fde337fe700 1 -- 172.21.15.77:0/1645965070 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fde38043ec0 con 0x7fde3c109b70 2024-05-27T08:27:06.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.016+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fde3c1a09b0 con 0x7fde3c109b70 2024-05-27T08:27:06.713 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.016+0000 7fde337fe700 1 -- 172.21.15.77:0/1645965070 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fde38032030 con 0x7fde3c109b70 2024-05-27T08:27:06.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.017+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 msgr2=0x7fde3c19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:06.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.017+0000 7fde4473f700 1 --2- 172.21.15.77:0/1645965070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c19cfb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde38009fb0 tx=0x7fde38011940 comp rx=0 tx=0).stop 2024-05-27T08:27:06.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.017+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 shutdown_connections 2024-05-27T08:27:06.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.017+0000 7fde4473f700 1 --2- 172.21.15.77:0/1645965070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fde3c109b70 0x7fde3c19cfb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:06.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.017+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 >> 172.21.15.77:0/1645965070 conn(0x7fde3c1034e0 msgr2=0x7fde3c1040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:06.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.017+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 shutdown_connections 2024-05-27T08:27:06.714 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:06.017+0000 7fde4473f700 1 -- 172.21.15.77:0/1645965070 wait complete. 2024-05-27T08:27:06.714 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (2/15)... 2024-05-27T08:27:07.129 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:06 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:06.747+0000 7f28c4926200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T08:27:07.129 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:06 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:06.841+0000 7f28c4926200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T08:27:07.565 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:07 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:07.210+0000 7f28c4926200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T08:27:07.566 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:07 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:07.564+0000 7f28c4926200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T08:27:07.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:07 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:07.658+0000 7f28c4926200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T08:27:08.379 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:07 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:07.903+0000 7f28c4926200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T08:27:08.379 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:08.037+0000 7f28c4926200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T08:27:08.379 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:08.134+0000 7f28c4926200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T08:27:08.822 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:08.476+0000 7f28c4926200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T08:27:08.823 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:08.569+0000 7f28c4926200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T08:27:09.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:09 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1282467473' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T08:27:09.379 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:09.198+0000 7f28c4926200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T08:27:09.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:09.419+0000 7f28c4926200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T08:27:10.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: Activating manager daemon x 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: mgrmap e2: x(active, starting, since 0.00328054s) 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: Manager daemon x is now available 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' 2024-05-27T08:27:10.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:10 smithi077 ceph-mon[32007]: from='mgr.14100 172.21.15.77:0/3299404973' entity='mgr.x' 2024-05-27T08:27:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:11 smithi077 ceph-mon[32007]: mgrmap e3: x(active, since 1.00764s) 2024-05-27T08:27:12.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:12 smithi077 ceph-mon[32007]: mgrmap e4: x(active, since 2s) 2024-05-27T08:27:14.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:27:14.918 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T08:27:14.919 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T08:27:14.920 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T08:27:14.927 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T08:27:14.927 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T08:27:14.928 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:14.929 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T08:26:55.342377+0000", 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.128+0000 7f7a39a90700 1 Processor -- start 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.128+0000 7f7a39a90700 1 -- start start 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.128+0000 7f7a39a90700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3410bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:14.930 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.128+0000 7f7a39a90700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a34076de0 con 0x7f7a34109b30 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.128+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3410bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.128+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3410bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52524/0 (socket says 172.21.15.77:52524) 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.128+0000 7f7a337fe700 1 -- 172.21.15.77:0/1577910622 learned_addr learned my addr 172.21.15.77:0/1577910622 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.129+0000 7f7a337fe700 1 -- 172.21.15.77:0/1577910622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3410c460 con 0x7f7a34109b30 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.129+0000 7f7a337fe700 1 --2- 172.21.15.77:0/1577910622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3410bf20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a280098a0 tx=0x7f7a28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d580090e3f5e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.129+0000 7f7a327fc700 1 -- 172.21.15.77:0/1577910622 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a28004160 con 0x7f7a34109b30 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.129+0000 7f7a327fc700 1 -- 172.21.15.77:0/1577910622 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7a28004300 con 0x7f7a34109b30 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.129+0000 7f7a327fc700 1 -- 172.21.15.77:0/1577910622 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a28011af0 con 0x7f7a34109b30 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.130+0000 7f7a39a90700 1 -- 172.21.15.77:0/1577910622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 msgr2=0x7f7a3410bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.130+0000 7f7a39a90700 1 --2- 172.21.15.77:0/1577910622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3410bf20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a280098a0 tx=0x7f7a28009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.130+0000 7f7a39a90700 1 -- 172.21.15.77:0/1577910622 shutdown_connections 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.130+0000 7f7a39a90700 1 --2- 172.21.15.77:0/1577910622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3410bf20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:14.931 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.130+0000 7f7a39a90700 1 -- 172.21.15.77:0/1577910622 >> 172.21.15.77:0/1577910622 conn(0x7f7a341034e0 msgr2=0x7f7a34105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.130+0000 7f7a39a90700 1 -- 172.21.15.77:0/1577910622 shutdown_connections 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.130+0000 7f7a39a90700 1 -- 172.21.15.77:0/1577910622 wait complete. 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.130+0000 7f7a39a90700 1 Processor -- start 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.131+0000 7f7a39a90700 1 -- start start 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.131+0000 7f7a39a90700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.131+0000 7f7a39a90700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3410d3d0 con 0x7f7a34109b30 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.131+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.131+0000 7f7a337fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52526/0 (socket says 172.21.15.77:52526) 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.131+0000 7f7a337fe700 1 -- 172.21.15.77:0/1282467473 learned_addr learned my addr 172.21.15.77:0/1282467473 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.131+0000 7f7a337fe700 1 -- 172.21.15.77:0/1282467473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3419d510 con 0x7f7a34109b30 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.131+0000 7f7a337fe700 1 --2- 172.21.15.77:0/1282467473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3419cfd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a28011ac0 tx=0x7f7a28011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.132+0000 7f7a30ff9700 1 -- 172.21.15.77:0/1282467473 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a28003970 con 0x7f7a34109b30 2024-05-27T08:27:14.932 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.132+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3419d7a0 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.132+0000 7f7a30ff9700 1 -- 172.21.15.77:0/1282467473 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7a28003b10 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.132+0000 7f7a30ff9700 1 -- 172.21.15.77:0/1282467473 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a28024610 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.132+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a341a0490 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.132+0000 7f7a30ff9700 1 -- 172.21.15.77:0/1282467473 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a28024c20 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.132+0000 7f7a30ff9700 1 -- 172.21.15.77:0/1282467473 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7a28019980 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.133+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a34064ba0 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.135+0000 7f7a30ff9700 1 -- 172.21.15.77:0/1282467473 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7a28043ec0 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.210+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7a341a0660 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.210+0000 7f7a30ff9700 1 -- 172.21.15.77:0/1282467473 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7a28032030 con 0x7f7a34109b30 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.211+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 msgr2=0x7f7a3419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.211+0000 7f7a39a90700 1 --2- 172.21.15.77:0/1282467473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3419cfd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a28011ac0 tx=0x7f7a28011ba0 comp rx=0 tx=0).stop 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.212+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 shutdown_connections 2024-05-27T08:27:14.933 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.212+0000 7f7a39a90700 1 --2- 172.21.15.77:0/1282467473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a34109b30 0x7f7a3419cfd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:14.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.212+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 >> 172.21.15.77:0/1282467473 conn(0x7f7a341034e0 msgr2=0x7f7a34104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:14.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.212+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 shutdown_connections 2024-05-27T08:27:14.934 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:09.212+0000 7f7a39a90700 1 -- 172.21.15.77:0/1282467473 wait complete. 2024-05-27T08:27:14.934 INFO:teuthology.orchestra.run.smithi077.stdout:mgr not available, waiting (3/15)... 2024-05-27T08:27:20.755 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:20 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2143619667' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-27T08:27:21.010 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "health": { 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 0 2024-05-27T08:27:21.011 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "a" 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "quorum_age": 20, 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:21.012 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-27T08:27:21.013 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-27T08:27:21.013 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-27T08:27:21.021 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-27T08:27:21.021 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-27T08:27:21.021 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-27T08:27:21.021 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-27T08:27:21.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "restful" 2024-05-27T08:27:21.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ], 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "modified": "2024-05-27T08:26:55.342377+0000", 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout }, 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.137+0000 7f24b4d53700 1 Processor -- start 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.137+0000 7f24b4d53700 1 -- start start 2024-05-27T08:27:21.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.138+0000 7f24b4d53700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b01094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.138+0000 7f24b4d53700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f24b01099e0 con 0x7f24b0109080 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.138+0000 7f24ae59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b01094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.138+0000 7f24ae59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b01094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54424/0 (socket says 172.21.15.77:54424) 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.138+0000 7f24ae59c700 1 -- 172.21.15.77:0/2535141793 learned_addr learned my addr 172.21.15.77:0/2535141793 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.138+0000 7f24ae59c700 1 -- 172.21.15.77:0/2535141793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b0109b60 con 0x7f24b0109080 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24ae59c700 1 --2- 172.21.15.77:0/2535141793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b01094a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24a00098a0 tx=0x7f24a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ede8e01cf028115 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24ad59a700 1 -- 172.21.15.77:0/2535141793 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a0004160 con 0x7f24b0109080 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24ad59a700 1 -- 172.21.15.77:0/2535141793 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24a0004300 con 0x7f24b0109080 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24ad59a700 1 -- 172.21.15.77:0/2535141793 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a0011af0 con 0x7f24b0109080 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24b4d53700 1 -- 172.21.15.77:0/2535141793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 msgr2=0x7f24b01094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:21.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24b4d53700 1 --2- 172.21.15.77:0/2535141793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b01094a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24a00098a0 tx=0x7f24a0009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24b4d53700 1 -- 172.21.15.77:0/2535141793 shutdown_connections 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24b4d53700 1 --2- 172.21.15.77:0/2535141793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b01094a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24b4d53700 1 -- 172.21.15.77:0/2535141793 >> 172.21.15.77:0/2535141793 conn(0x7f24b0104720 msgr2=0x7f24b0106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24b4d53700 1 -- 172.21.15.77:0/2535141793 shutdown_connections 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.139+0000 7f24b4d53700 1 -- 172.21.15.77:0/2535141793 wait complete. 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.140+0000 7f24b4d53700 1 Processor -- start 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.140+0000 7f24b4d53700 1 -- start start 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.140+0000 7f24b4d53700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.140+0000 7f24b4d53700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f24b010b080 con 0x7f24b0109080 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f24ae59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f24ae59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54432/0 (socket says 172.21.15.77:54432) 2024-05-27T08:27:21.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f24ae59c700 1 -- 172.21.15.77:0/2143619667 learned_addr learned my addr 172.21.15.77:0/2143619667 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f24ae59c700 1 -- 172.21.15.77:0/2143619667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b019d4a0 con 0x7f24b0109080 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f24ae59c700 1 --2- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b019cf60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24a0011ac0 tx=0x7f24a0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f249f7fe700 1 -- 172.21.15.77:0/2143619667 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a0003b20 con 0x7f24b0109080 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f249f7fe700 1 -- 172.21.15.77:0/2143619667 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24a001a8c0 con 0x7f24b0109080 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b019d730 con 0x7f24b0109080 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f249f7fe700 1 -- 172.21.15.77:0/2143619667 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24a00248d0 con 0x7f24b0109080 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.141+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b01a0420 con 0x7f24b0109080 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.142+0000 7f249f7fe700 1 -- 172.21.15.77:0/2143619667 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f24a0022020 con 0x7f24b0109080 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.142+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b0064ba0 con 0x7f24b0109080 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.142+0000 7f249f7fe700 1 --2- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f249803e470 0x7f2498040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:21.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.142+0000 7f249f7fe700 1 -- 172.21.15.77:0/2143619667 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f24a0053590 con 0x7f24b0109080 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.143+0000 7f24add9b700 1 --2- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f249803e470 0x7f2498040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.143+0000 7f24add9b700 1 --2- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f249803e470 0x7f2498040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006fd0 tx=0x7f24a4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.148+0000 7f249f7fe700 1 -- 172.21.15.77:0/2143619667 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f24a001ade0 con 0x7f24b0109080 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.441+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f24b01a05f0 con 0x7f24b0109080 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.441+0000 7f249f7fe700 1 -- 172.21.15.77:0/2143619667 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f24b01a05f0 con 0x7f24b0109080 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f249803e470 msgr2=0x7f2498040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 --2- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f249803e470 0x7f2498040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006fd0 tx=0x7f24a4006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 msgr2=0x7f24b019cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 --2- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b019cf60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24a0011ac0 tx=0x7f24a0011ba0 comp rx=0 tx=0).stop 2024-05-27T08:27:21.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 shutdown_connections 2024-05-27T08:27:21.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 --2- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f249803e470 0x7f2498040930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:21.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 --2- 172.21.15.77:0/2143619667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f24b0109080 0x7f24b019cf60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:21.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 >> 172.21.15.77:0/2143619667 conn(0x7f24b0104720 msgr2=0x7f24b0105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:21.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 shutdown_connections 2024-05-27T08:27:21.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:20.445+0000 7f24b4d53700 1 -- 172.21.15.77:0/2143619667 wait complete. 2024-05-27T08:27:21.029 INFO:teuthology.orchestra.run.smithi077.stdout:mgr is available 2024-05-27T08:27:21.945 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:21 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2120124343' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [global] 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout fsid = c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:27:22.229 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout [osd] 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.422+0000 7fb75e568700 1 Processor -- start 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.422+0000 7fb75e568700 1 -- start start 2024-05-27T08:27:22.230 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.422+0000 7fb75e568700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75810bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.422+0000 7fb75e568700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb75807ce40 con 0x7fb758109b60 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.422+0000 7fb757fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75810bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.422+0000 7fb757fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75810bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54444/0 (socket says 172.21.15.77:54444) 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.422+0000 7fb757fff700 1 -- 172.21.15.77:0/4036222956 learned_addr learned my addr 172.21.15.77:0/4036222956 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.422+0000 7fb757fff700 1 -- 172.21.15.77:0/4036222956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75810c490 con 0x7fb758109b60 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb757fff700 1 --2- 172.21.15.77:0/4036222956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75810bf50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb7480098a0 tx=0x7fb748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483adcfd001aa8d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb756ffd700 1 -- 172.21.15.77:0/4036222956 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb748004160 con 0x7fb758109b60 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb756ffd700 1 -- 172.21.15.77:0/4036222956 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb748004300 con 0x7fb758109b60 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb756ffd700 1 -- 172.21.15.77:0/4036222956 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb748011af0 con 0x7fb758109b60 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb75e568700 1 -- 172.21.15.77:0/4036222956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 msgr2=0x7fb75810bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb75e568700 1 --2- 172.21.15.77:0/4036222956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75810bf50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb7480098a0 tx=0x7fb748009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:22.231 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb75e568700 1 -- 172.21.15.77:0/4036222956 shutdown_connections 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb75e568700 1 --2- 172.21.15.77:0/4036222956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75810bf50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb75e568700 1 -- 172.21.15.77:0/4036222956 >> 172.21.15.77:0/4036222956 conn(0x7fb758103570 msgr2=0x7fb758105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb75e568700 1 -- 172.21.15.77:0/4036222956 shutdown_connections 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.423+0000 7fb75e568700 1 -- 172.21.15.77:0/4036222956 wait complete. 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.424+0000 7fb75e568700 1 Processor -- start 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.424+0000 7fb75e568700 1 -- start start 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.424+0000 7fb75e568700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.424+0000 7fb75e568700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb75810d400 con 0x7fb758109b60 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.424+0000 7fb757fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb757fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54456/0 (socket says 172.21.15.77:54456) 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb757fff700 1 -- 172.21.15.77:0/2120124343 learned_addr learned my addr 172.21.15.77:0/2120124343 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:22.232 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb757fff700 1 -- 172.21.15.77:0/2120124343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75819d520 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb757fff700 1 --2- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75819cfe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb748011ac0 tx=0x7fb748011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb7557fa700 1 -- 172.21.15.77:0/2120124343 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb748003970 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb7557fa700 1 -- 172.21.15.77:0/2120124343 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb748003b10 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75819d7b0 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb7557fa700 1 -- 172.21.15.77:0/2120124343 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb748024610 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.425+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581a04a0 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.426+0000 7fb7557fa700 1 -- 172.21.15.77:0/2120124343 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fb748024890 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.426+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb758064ba0 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.426+0000 7fb7557fa700 1 --2- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7fb74003e420 0x7fb7400408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.426+0000 7fb7557fa700 1 -- 172.21.15.77:0/2120124343 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb748053820 con 0x7fb758109b60 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.426+0000 7fb7577fe700 1 --2- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7fb74003e420 0x7fb7400408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:22.233 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.427+0000 7fb7577fe700 1 --2- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7fb74003e420 0x7fb7400408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb74c006fd0 tx=0x7fb74c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.431+0000 7fb7557fa700 1 -- 172.21.15.77:0/2120124343 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb74802c350 con 0x7fb758109b60 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.632+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb7581a0670 con 0x7fb758109b60 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.633+0000 7fb7557fa700 1 -- 172.21.15.77:0/2120124343 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fb7581a0670 con 0x7fb758109b60 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.635+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7fb74003e420 msgr2=0x7fb7400408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.635+0000 7fb75e568700 1 --2- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7fb74003e420 0x7fb7400408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb74c006fd0 tx=0x7fb74c006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.635+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 msgr2=0x7fb75819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.635+0000 7fb75e568700 1 --2- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75819cfe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb748011ac0 tx=0x7fb748011ba0 comp rx=0 tx=0).stop 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.636+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 shutdown_connections 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.636+0000 7fb75e568700 1 --2- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7fb74003e420 0x7fb7400408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.636+0000 7fb75e568700 1 --2- 172.21.15.77:0/2120124343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb758109b60 0x7fb75819cfe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.636+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 >> 172.21.15.77:0/2120124343 conn(0x7fb758103570 msgr2=0x7fb7581040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:22.234 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.636+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 shutdown_connections 2024-05-27T08:27:22.235 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:21.636+0000 7fb75e568700 1 -- 172.21.15.77:0/2120124343 wait complete. 2024-05-27T08:27:22.235 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling cephadm module... 2024-05-27T08:27:23.302 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:22 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/4186804288' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-27T08:27:23.302 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:23 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: ignoring --setuser ceph since I am not root 2024-05-27T08:27:23.302 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:23 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: ignoring --setgroup ceph since I am not root 2024-05-27T08:27:23.302 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:23 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:23.144+0000 7fad96c00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T08:27:23.630 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:23 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:23.332+0000 7fad96c00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:27:23.630 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:23 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:23.435+0000 7fad96c00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T08:27:23.900 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.655+0000 7ff7c977e700 1 Processor -- start 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.655+0000 7ff7c977e700 1 -- start start 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c977e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c410bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c977e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c407cea0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c410bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c410bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54462/0 (socket says 172.21.15.77:54462) 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c2ffd700 1 -- 172.21.15.77:0/4171339519 learned_addr learned my addr 172.21.15.77:0/4171339519 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c2ffd700 1 -- 172.21.15.77:0/4171339519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c410c410 con 0x7ff7c4109ae0 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c2ffd700 1 --2- 172.21.15.77:0/4171339519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c410bed0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098a0 tx=0x7ff7ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14acfc8aff3c780a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c1ffb700 1 -- 172.21.15.77:0/4171339519 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7ac004160 con 0x7ff7c4109ae0 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c1ffb700 1 -- 172.21.15.77:0/4171339519 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff7ac004300 con 0x7ff7c4109ae0 2024-05-27T08:27:23.901 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.656+0000 7ff7c1ffb700 1 -- 172.21.15.77:0/4171339519 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7ac011af0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.657+0000 7ff7c977e700 1 -- 172.21.15.77:0/4171339519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 msgr2=0x7ff7c410bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.657+0000 7ff7c977e700 1 --2- 172.21.15.77:0/4171339519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c410bed0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0098a0 tx=0x7ff7ac009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.657+0000 7ff7c977e700 1 -- 172.21.15.77:0/4171339519 shutdown_connections 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.657+0000 7ff7c977e700 1 --2- 172.21.15.77:0/4171339519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c410bed0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.657+0000 7ff7c977e700 1 -- 172.21.15.77:0/4171339519 >> 172.21.15.77:0/4171339519 conn(0x7ff7c4103510 msgr2=0x7ff7c4105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.657+0000 7ff7c977e700 1 -- 172.21.15.77:0/4171339519 shutdown_connections 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.657+0000 7ff7c977e700 1 -- 172.21.15.77:0/4171339519 wait complete. 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c977e700 1 Processor -- start 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c977e700 1 -- start start 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c977e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c977e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c410d3e0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:23.902 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54466/0 (socket says 172.21.15.77:54466) 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c2ffd700 1 -- 172.21.15.77:0/4186804288 learned_addr learned my addr 172.21.15.77:0/4186804288 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c2ffd700 1 -- 172.21.15.77:0/4186804288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c419d550 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.658+0000 7ff7c2ffd700 1 --2- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c419d010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009fb0 tx=0x7ff7ac011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.659+0000 7ff7bbfff700 1 -- 172.21.15.77:0/4186804288 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7ac0036a0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.659+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c419d7e0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.659+0000 7ff7bbfff700 1 -- 172.21.15.77:0/4186804288 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff7ac01a430 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.659+0000 7ff7bbfff700 1 -- 172.21.15.77:0/4186804288 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7ac019550 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.659+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c41a04d0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.660+0000 7ff7bbfff700 1 -- 172.21.15.77:0/4186804288 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7ff7ac019850 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.660+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c4064ba0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.660+0000 7ff7bbfff700 1 --2- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7ff7b003e3d0 0x7ff7b0040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.660+0000 7ff7bbfff700 1 -- 172.21.15.77:0/4186804288 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff7ac0537d0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.903 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.660+0000 7ff7c27fc700 1 --2- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7ff7b003e3d0 0x7ff7b0040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.660+0000 7ff7c27fc700 1 --2- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7ff7b003e3d0 0x7ff7b0040890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4006fd0 tx=0x7ff7b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.665+0000 7ff7bbfff700 1 -- 172.21.15.77:0/4186804288 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7ac017e90 con 0x7ff7c4109ae0 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.922+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff7c41a06a0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.974+0000 7ff7bbfff700 1 -- 172.21.15.77:0/4186804288 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff7c41a06a0 con 0x7ff7c4109ae0 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.975+0000 7ff7bbfff700 1 -- 172.21.15.77:0/4186804288 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7ff7ac00fd80 con 0x7ff7c4109ae0 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7ff7b003e3d0 msgr2=0x7ff7b0040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 --2- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7ff7b003e3d0 0x7ff7b0040890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4006fd0 tx=0x7ff7b4006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 msgr2=0x7ff7c419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 --2- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c419d010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009fb0 tx=0x7ff7ac011940 comp rx=0 tx=0).stop 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 shutdown_connections 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 --2- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7ff7b003e3d0 0x7ff7b0040890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 --2- 172.21.15.77:0/4186804288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7c4109ae0 0x7ff7c419d010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:23.904 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 >> 172.21.15.77:0/4186804288 conn(0x7ff7c4103510 msgr2=0x7ff7c4104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:23.905 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 shutdown_connections 2024-05-27T08:27:23.905 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:22.977+0000 7ff7c977e700 1 -- 172.21.15.77:0/4186804288 wait complete. 2024-05-27T08:27:24.012 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:23 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/4186804288' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-27T08:27:24.012 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:23 smithi077 ceph-mon[32007]: mgrmap e5: x(active, since 13s) 2024-05-27T08:27:24.012 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:23 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:23.746+0000 7fad96c00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T08:27:24.012 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:23 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:23.831+0000 7fad96c00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T08:27:25.186 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:24 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2560311155' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T08:27:25.186 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:25.043+0000 7fad96c00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T08:27:25.188 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-05-27T08:27:25.193 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-27T08:27:25.193 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T08:27:25.193 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T08:27:25.193 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.331+0000 7f805edac700 1 Processor -- start 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.332+0000 7f805edac700 1 -- start start 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.332+0000 7f805edac700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f8058109910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.332+0000 7f805edac700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8058109ee0 con 0x7f80581094f0 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.332+0000 7f805cb48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f8058109910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.332+0000 7f805cb48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f8058109910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54490/0 (socket says 172.21.15.77:54490) 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.332+0000 7f805cb48700 1 -- 172.21.15.77:0/4175704303 learned_addr learned my addr 172.21.15.77:0/4175704303 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.332+0000 7f805cb48700 1 -- 172.21.15.77:0/4175704303 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8058108150 con 0x7f80581094f0 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.332+0000 7f805cb48700 1 --2- 172.21.15.77:0/4175704303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f8058109910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f804c0098a0 tx=0x7f804c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fdb5a5ed4bc574 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:25.194 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f80577fe700 1 -- 172.21.15.77:0/4175704303 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c004160 con 0x7f80581094f0 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f80577fe700 1 -- 172.21.15.77:0/4175704303 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f804c004300 con 0x7f80581094f0 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f80577fe700 1 -- 172.21.15.77:0/4175704303 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c011af0 con 0x7f80581094f0 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f805edac700 1 -- 172.21.15.77:0/4175704303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 msgr2=0x7f8058109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f805edac700 1 --2- 172.21.15.77:0/4175704303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f8058109910 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f804c0098a0 tx=0x7f804c009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f805edac700 1 -- 172.21.15.77:0/4175704303 shutdown_connections 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f805edac700 1 --2- 172.21.15.77:0/4175704303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f8058109910 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f805edac700 1 -- 172.21.15.77:0/4175704303 >> 172.21.15.77:0/4175704303 conn(0x7f8058102c80 msgr2=0x7f80581050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f805edac700 1 -- 172.21.15.77:0/4175704303 shutdown_connections 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.333+0000 7f805edac700 1 -- 172.21.15.77:0/4175704303 wait complete. 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.334+0000 7f805edac700 1 Processor -- start 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.334+0000 7f805edac700 1 -- start start 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.334+0000 7f805edac700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f805819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:25.195 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.334+0000 7f805edac700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f805810c510 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.334+0000 7f805cb48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f805819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.334+0000 7f805cb48700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f805819cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54498/0 (socket says 172.21.15.77:54498) 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.334+0000 7f805cb48700 1 -- 172.21.15.77:0/2560311155 learned_addr learned my addr 172.21.15.77:0/2560311155 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.335+0000 7f805cb48700 1 -- 172.21.15.77:0/2560311155 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805819d530 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.335+0000 7f805cb48700 1 --2- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f805819cff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f804c011ac0 tx=0x7f804c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.335+0000 7f8055ffb700 1 -- 172.21.15.77:0/2560311155 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c003970 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.335+0000 7f8055ffb700 1 -- 172.21.15.77:0/2560311155 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f804c003b10 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.335+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805819d7c0 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.335+0000 7f8055ffb700 1 -- 172.21.15.77:0/2560311155 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c024610 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.335+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80581a04b0 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.336+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8058064ba0 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.337+0000 7f8055ffb700 1 -- 172.21.15.77:0/2560311155 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f804c01a9e0 con 0x7f80581094f0 2024-05-27T08:27:25.196 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.337+0000 7f8055ffb700 1 --2- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f804803e470 0x7f8048040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.337+0000 7f8055ffb700 1 -- 172.21.15.77:0/2560311155 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f804c053890 con 0x7f80581094f0 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.337+0000 7f8057fff700 1 -- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f804803e470 msgr2=0x7f8048040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2203620437 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.337+0000 7f8057fff700 1 --2- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f804803e470 0x7f8048040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.342+0000 7f8055ffb700 1 -- 172.21.15.77:0/2560311155 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f804c0532b0 con 0x7f80581094f0 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.538+0000 7f8057fff700 1 -- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f804803e470 msgr2=0x7f8048040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2203620437 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.538+0000 7f8057fff700 1 --2- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f804803e470 0x7f8048040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.606+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f80580008d0 con 0x7f80581094f0 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.607+0000 7f8055ffb700 1 -- 172.21.15.77:0/2560311155 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f804c024c10 con 0x7f80581094f0 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.609+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f804803e470 msgr2=0x7f8048040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.609+0000 7f805edac700 1 --2- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f804803e470 0x7f8048040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.609+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 msgr2=0x7f805819cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.609+0000 7f805edac700 1 --2- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f805819cff0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f804c011ac0 tx=0x7f804c011ba0 comp rx=0 tx=0).stop 2024-05-27T08:27:25.197 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.610+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 shutdown_connections 2024-05-27T08:27:25.198 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.610+0000 7f805edac700 1 --2- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f804803e470 0x7f8048040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:25.198 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.610+0000 7f805edac700 1 --2- 172.21.15.77:0/2560311155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80581094f0 0x7f805819cff0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:25.198 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.610+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 >> 172.21.15.77:0/2560311155 conn(0x7f8058102c80 msgr2=0x7f8058103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:25.198 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.610+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 shutdown_connections 2024-05-27T08:27:25.198 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:24.610+0000 7f805edac700 1 -- 172.21.15.77:0/2560311155 wait complete. 2024-05-27T08:27:25.198 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2024-05-27T08:27:25.198 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 5... 2024-05-27T08:27:25.438 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:25.219+0000 7fad96c00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T08:27:25.439 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:25.389+0000 7fad96c00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T08:27:25.984 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:25.690+0000 7fad96c00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T08:27:25.985 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:25.783+0000 7fad96c00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T08:27:25.985 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:25.983+0000 7fad96c00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T08:27:26.237 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:26 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:26.182+0000 7fad96c00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:27:27.002 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:26 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:26.782+0000 7fad96c00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T08:27:27.002 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:27 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:27.001+0000 7fad96c00200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T08:27:27.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:27 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:27.420+0000 7fad96c00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T08:27:27.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:27 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:27.515+0000 7fad96c00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T08:27:28.230 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:27 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:27.878+0000 7fad96c00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T08:27:28.230 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:28.229+0000 7fad96c00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T08:27:28.561 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:28.321+0000 7fad96c00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T08:27:28.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:28.560+0000 7fad96c00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T08:27:28.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:28.693+0000 7fad96c00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T08:27:28.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:28.788+0000 7fad96c00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T08:27:29.379 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:29.110+0000 7fad96c00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T08:27:29.379 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:29.201+0000 7fad96c00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T08:27:30.064 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:29.796+0000 7fad96c00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T08:27:30.064 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:30 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:30.010+0000 7fad96c00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T08:27:30.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: Active manager daemon x restarted 2024-05-27T08:27:30.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: Activating manager daemon x 2024-05-27T08:27:30.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: osdmap e2: 0 total, 0 up, 0 in 2024-05-27T08:27:30.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: mgrmap e6: x(active, starting, since 0.00369979s) 2024-05-27T08:27:30.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: Manager daemon x is now available 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:27:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:30 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:27:31.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:31 smithi077 ceph-mon[32007]: Found migration_current of "None". Setting to last migration. 2024-05-27T08:27:31.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:31 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T08:27:31.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:31 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T08:27:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:31 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:31 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:31 smithi077 ceph-mon[32007]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T08:27:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:31 smithi077 ceph-mon[32007]: mgrmap e7: x(active, since 1.00569s) 2024-05-27T08:27:31.651 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.629+0000 7f1672c89700 1 Processor -- start 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.629+0000 7f1672c89700 1 -- start start 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f1672c89700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c107da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f1672c89700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f166c07ce20 con 0x7f166c107980 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f1670a25700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f1670a25700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54514/0 (socket says 172.21.15.77:54514) 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f1670a25700 1 -- 172.21.15.77:0/3493086550 learned_addr learned my addr 172.21.15.77:0/3493086550 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:31.652 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f1670a25700 1 -- 172.21.15.77:0/3493086550 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c1082e0 con 0x7f166c107980 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f1670a25700 1 --2- 172.21.15.77:0/3493086550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c107da0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16600098a0 tx=0x7f1660009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f7653f3a564356 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f166b7fe700 1 -- 172.21.15.77:0/3493086550 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1660004160 con 0x7f166c107980 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f166b7fe700 1 -- 172.21.15.77:0/3493086550 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1660004300 con 0x7f166c107980 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.630+0000 7f166b7fe700 1 -- 172.21.15.77:0/3493086550 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1660011af0 con 0x7f166c107980 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.631+0000 7f1672c89700 1 -- 172.21.15.77:0/3493086550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 msgr2=0x7f166c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.631+0000 7f1672c89700 1 --2- 172.21.15.77:0/3493086550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c107da0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16600098a0 tx=0x7f1660009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.631+0000 7f1672c89700 1 -- 172.21.15.77:0/3493086550 shutdown_connections 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.631+0000 7f1672c89700 1 --2- 172.21.15.77:0/3493086550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c107da0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.631+0000 7f1672c89700 1 -- 172.21.15.77:0/3493086550 >> 172.21.15.77:0/3493086550 conn(0x7f166c103520 msgr2=0x7f166c105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.631+0000 7f1672c89700 1 -- 172.21.15.77:0/3493086550 shutdown_connections 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.631+0000 7f1672c89700 1 -- 172.21.15.77:0/3493086550 wait complete. 2024-05-27T08:27:31.653 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.632+0000 7f1672c89700 1 Processor -- start 2024-05-27T08:27:31.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.632+0000 7f1672c89700 1 -- start start 2024-05-27T08:27:31.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.632+0000 7f1672c89700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:31.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.632+0000 7f1672c89700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f166c109310 con 0x7f166c107980 2024-05-27T08:27:31.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.632+0000 7f1670a25700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:31.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.632+0000 7f1670a25700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c19f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54522/0 (socket says 172.21.15.77:54522) 2024-05-27T08:27:31.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.632+0000 7f1670a25700 1 -- 172.21.15.77:0/49487622 learned_addr learned my addr 172.21.15.77:0/49487622 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:31.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.633+0000 7f1670a25700 1 -- 172.21.15.77:0/49487622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166c19f750 con 0x7f166c107980 2024-05-27T08:27:31.654 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.633+0000 7f1670a25700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c19f210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1660009fb0 tx=0x7f1660011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.633+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16600036a0 con 0x7f166c107980 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.633+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f166001a430 con 0x7f166c107980 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.633+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1660019550 con 0x7f166c107980 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.633+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166c19f9e0 con 0x7f166c107980 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.633+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f166c1a26d0 con 0x7f166c107980 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.634+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f166001a8e0 con 0x7f166c107980 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.634+0000 7f1669ffb700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 0x7f165c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.634+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 --> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f165c041040 con 0x7f165c03e470 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.634+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f16600537f0 con 0x7f166c107980 2024-05-27T08:27:31.655 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.634+0000 7f166bfff700 1 -- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 msgr2=0x7f165c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2203620437 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.634+0000 7f166bfff700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 0x7f165c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.835+0000 7f166bfff700 1 -- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 msgr2=0x7f165c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2203620437 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:25.835+0000 7f166bfff700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 0x7f165c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:26.236+0000 7f166bfff700 1 -- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 msgr2=0x7f165c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2203620437 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:26.236+0000 7f166bfff700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 0x7f165c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:27.037+0000 7f166bfff700 1 -- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 msgr2=0x7f165c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2203620437 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:27.037+0000 7f166bfff700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 0x7f165c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:28.638+0000 7f166bfff700 1 -- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 msgr2=0x7f165c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2203620437 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:28.638+0000 7f166bfff700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 0x7f165c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:30.013+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f1660003840 con 0x7f166c107980 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:30.014+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 msgr2=0x7f165c040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:27:31.656 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:30.014+0000 7f1669ffb700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 0x7f165c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.015+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f16600199f0 con 0x7f166c107980 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.015+0000 7f1669ffb700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f165c041d60 0x7f165c044150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.015+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1660018e50 con 0x7f165c041d60 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.015+0000 7f166bfff700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f165c041d60 0x7f165c044150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.016+0000 7f166bfff700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f165c041d60 0x7f165c044150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1658003a10 tx=0x7f16580092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.017+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mgr.14120 v2:172.21.15.77:6800/2456305222 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f1660018e50 con 0x7f165c041d60 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f166c06a6a0 con 0x7f165c041d60 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1669ffb700 1 -- 172.21.15.77:0/49487622 <== mgr.14120 v2:172.21.15.77:6800/2456305222 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f166c06a6a0 con 0x7f165c041d60 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f165c041d60 msgr2=0x7f165c044150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f165c041d60 0x7f165c044150 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1658003a10 tx=0x7f16580092b0 comp rx=0 tx=0).stop 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 msgr2=0x7f166c19f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:31.657 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c19f210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1660009fb0 tx=0x7f1660011940 comp rx=0 tx=0).stop 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 shutdown_connections 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f165c041d60 0x7f165c044150 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f166c107980 0x7f166c19f210 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 --2- 172.21.15.77:0/49487622 >> [v2:172.21.15.77:6800/2203620437,v1:172.21.15.77:6801/2203620437] conn(0x7f165c03e470 0x7f165c040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 >> 172.21.15.77:0/49487622 conn(0x7f166c103520 msgr2=0x7f166c195e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 shutdown_connections 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:31.031+0000 7f1672c89700 1 -- 172.21.15.77:0/49487622 wait complete. 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 5 is available 2024-05-27T08:27:31.658 INFO:teuthology.orchestra.run.smithi077.stdout:Setting orchestrator backend to cephadm... 2024-05-27T08:27:32.297 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:32 smithi077 ceph-mon[32007]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T08:27:32.297 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:32 smithi077 ceph-mon[32007]: [27/May/2024:08:27:31] ENGINE Bus STARTING 2024-05-27T08:27:32.297 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:32 smithi077 ceph-mon[32007]: [27/May/2024:08:27:31] ENGINE Serving on http://172.21.15.77:8765 2024-05-27T08:27:32.297 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:32 smithi077 ceph-mon[32007]: [27/May/2024:08:27:31] ENGINE Serving on https://172.21.15.77:7150 2024-05-27T08:27:32.297 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:32 smithi077 ceph-mon[32007]: [27/May/2024:08:27:31] ENGINE Bus STARTED 2024-05-27T08:27:32.297 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:32 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:27:32.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.062+0000 7f85d9176700 1 Processor -- start 2024-05-27T08:27:32.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.062+0000 7f85d9176700 1 -- start start 2024-05-27T08:27:32.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.062+0000 7f85d9176700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d4107710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:32.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.062+0000 7f85d9176700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85d4107ce0 con 0x7f85d41072f0 2024-05-27T08:27:32.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.062+0000 7f85d2d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d4107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:32.830 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.062+0000 7f85d2d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d4107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55280/0 (socket says 172.21.15.77:55280) 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.062+0000 7f85d2d9d700 1 -- 172.21.15.77:0/3344641107 learned_addr learned my addr 172.21.15.77:0/3344641107 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.062+0000 7f85d2d9d700 1 -- 172.21.15.77:0/3344641107 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d410a2d0 con 0x7f85d41072f0 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d2d9d700 1 --2- 172.21.15.77:0/3344641107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d4107710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85c40098a0 tx=0x7f85c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a4b7b4b9a2303d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d1d9b700 1 -- 172.21.15.77:0/3344641107 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c4004160 con 0x7f85d41072f0 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d1d9b700 1 -- 172.21.15.77:0/3344641107 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f85c4004300 con 0x7f85d41072f0 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d1d9b700 1 -- 172.21.15.77:0/3344641107 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c4011af0 con 0x7f85d41072f0 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d9176700 1 -- 172.21.15.77:0/3344641107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 msgr2=0x7f85d4107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d9176700 1 --2- 172.21.15.77:0/3344641107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d4107710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85c40098a0 tx=0x7f85c4009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d9176700 1 -- 172.21.15.77:0/3344641107 shutdown_connections 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d9176700 1 --2- 172.21.15.77:0/3344641107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d4107710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:32.831 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d9176700 1 -- 172.21.15.77:0/3344641107 >> 172.21.15.77:0/3344641107 conn(0x7f85d4102d10 msgr2=0x7f85d4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d9176700 1 -- 172.21.15.77:0/3344641107 shutdown_connections 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.063+0000 7f85d9176700 1 -- 172.21.15.77:0/3344641107 wait complete. 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.064+0000 7f85d9176700 1 Processor -- start 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.064+0000 7f85d9176700 1 -- start start 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.065+0000 7f85d9176700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d419d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.065+0000 7f85d9176700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85d410d680 con 0x7f85d41072f0 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.065+0000 7f85d2d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d419d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.065+0000 7f85d2d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d419d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55284/0 (socket says 172.21.15.77:55284) 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.065+0000 7f85d2d9d700 1 -- 172.21.15.77:0/411892629 learned_addr learned my addr 172.21.15.77:0/411892629 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.065+0000 7f85d2d9d700 1 -- 172.21.15.77:0/411892629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d419d580 con 0x7f85d41072f0 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.065+0000 7f85d2d9d700 1 --2- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d419d040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009fb0 tx=0x7f85c4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:32.832 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.066+0000 7f85bbfff700 1 -- 172.21.15.77:0/411892629 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c40036a0 con 0x7f85d41072f0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.066+0000 7f85bbfff700 1 -- 172.21.15.77:0/411892629 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f85c401a430 con 0x7f85d41072f0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.066+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d419d810 con 0x7f85d41072f0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.066+0000 7f85bbfff700 1 -- 172.21.15.77:0/411892629 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c4019550 con 0x7f85d41072f0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.066+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d41a0500 con 0x7f85d41072f0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.067+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d4064ba0 con 0x7f85d41072f0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.070+0000 7f85bbfff700 1 -- 172.21.15.77:0/411892629 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f85c401a8e0 con 0x7f85d41072f0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.070+0000 7f85bbfff700 1 --2- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f85bc03e420 0x7f85bc0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.070+0000 7f85bbfff700 1 -- 172.21.15.77:0/411892629 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85c4053b80 con 0x7f85d41072f0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.070+0000 7f85d259c700 1 --2- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f85bc03e420 0x7f85bc0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.070+0000 7f85d259c700 1 --2- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f85bc03e420 0x7f85bc0408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f85c8006fd0 tx=0x7f85c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:32.833 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.075+0000 7f85bbfff700 1 -- 172.21.15.77:0/411892629 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f85c402de00 con 0x7f85d41072f0 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.296+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f85d4103fb0 con 0x7f85bc03e420 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.301+0000 7f85bbfff700 1 -- 172.21.15.77:0/411892629 <== mgr.14120 v2:172.21.15.77:6800/2456305222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f85d4103fb0 con 0x7f85bc03e420 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.303+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f85bc03e420 msgr2=0x7f85bc0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.303+0000 7f85d9176700 1 --2- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f85bc03e420 0x7f85bc0408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f85c8006fd0 tx=0x7f85c8006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.303+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 msgr2=0x7f85d419d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.303+0000 7f85d9176700 1 --2- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d419d040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009fb0 tx=0x7f85c4011940 comp rx=0 tx=0).stop 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.304+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 shutdown_connections 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.304+0000 7f85d9176700 1 --2- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f85bc03e420 0x7f85bc0408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.304+0000 7f85d9176700 1 --2- 172.21.15.77:0/411892629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85d41072f0 0x7f85d419d040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.304+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 >> 172.21.15.77:0/411892629 conn(0x7f85d4102d10 msgr2=0x7f85d41038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.304+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 shutdown_connections 2024-05-27T08:27:32.834 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:32.304+0000 7f85d9176700 1 -- 172.21.15.77:0/411892629 wait complete. 2024-05-27T08:27:33.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:33 smithi077 ceph-mon[32007]: mgrmap e8: x(active, since 2s) 2024-05-27T08:27:33.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:33 smithi077 ceph-mon[32007]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:27:33.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:33 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:33.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:33 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:27:33.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-27T08:27:33.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.266+0000 7f13598f2700 1 Processor -- start 2024-05-27T08:27:33.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f13598f2700 1 -- start start 2024-05-27T08:27:33.988 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f13598f2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f135410bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f13598f2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1354076d60 con 0x7f1354109b70 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f1352ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f135410bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f1352ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f135410bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55298/0 (socket says 172.21.15.77:55298) 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f1352ffd700 1 -- 172.21.15.77:0/2174656085 learned_addr learned my addr 172.21.15.77:0/2174656085 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f1352ffd700 1 -- 172.21.15.77:0/2174656085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135410c4a0 con 0x7f1354109b70 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f1352ffd700 1 --2- 172.21.15.77:0/2174656085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f135410bf60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f13440098a0 tx=0x7f1344009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ccc216c084a545 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.267+0000 7f13527fc700 1 -- 172.21.15.77:0/2174656085 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1344004160 con 0x7f1354109b70 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13527fc700 1 -- 172.21.15.77:0/2174656085 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1344004300 con 0x7f1354109b70 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13527fc700 1 -- 172.21.15.77:0/2174656085 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1344011af0 con 0x7f1354109b70 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13598f2700 1 -- 172.21.15.77:0/2174656085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 msgr2=0x7f135410bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13598f2700 1 --2- 172.21.15.77:0/2174656085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f135410bf60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f13440098a0 tx=0x7f1344009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:33.989 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13598f2700 1 -- 172.21.15.77:0/2174656085 shutdown_connections 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13598f2700 1 --2- 172.21.15.77:0/2174656085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f135410bf60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13598f2700 1 -- 172.21.15.77:0/2174656085 >> 172.21.15.77:0/2174656085 conn(0x7f1354103540 msgr2=0x7f1354105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13598f2700 1 -- 172.21.15.77:0/2174656085 shutdown_connections 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.268+0000 7f13598f2700 1 -- 172.21.15.77:0/2174656085 wait complete. 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f13598f2700 1 Processor -- start 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f13598f2700 1 -- start start 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f13598f2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f13541093f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f13598f2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f135410d3a0 con 0x7f1354109b70 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f1352ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f13541093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f1352ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f13541093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55310/0 (socket says 172.21.15.77:55310) 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f1352ffd700 1 -- 172.21.15.77:0/1666851203 learned_addr learned my addr 172.21.15.77:0/1666851203 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f1352ffd700 1 -- 172.21.15.77:0/1666851203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1354109930 con 0x7f1354109b70 2024-05-27T08:27:33.990 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.269+0000 7f1352ffd700 1 --2- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f13541093f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1344011ac0 tx=0x7f1344011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.270+0000 7f13588f0700 1 -- 172.21.15.77:0/1666851203 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1344003b20 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.270+0000 7f13588f0700 1 -- 172.21.15.77:0/1666851203 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f134401a8c0 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.270+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1354107b20 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.270+0000 7f13588f0700 1 -- 172.21.15.77:0/1666851203 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13440248d0 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.270+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1354108060 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.271+0000 7f13588f0700 1 -- 172.21.15.77:0/1666851203 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1344022020 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.271+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135406a6a0 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.271+0000 7f13588f0700 1 --2- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f134003e4c0 0x7f1340040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.271+0000 7f13588f0700 1 -- 172.21.15.77:0/1666851203 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f13440537d0 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.271+0000 7f134bfff700 1 --2- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f134003e4c0 0x7f1340040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.271+0000 7f134bfff700 1 --2- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f134003e4c0 0x7f1340040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f133c006fd0 tx=0x7f133c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.276+0000 7f13588f0700 1 -- 172.21.15.77:0/1666851203 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f134400e260 con 0x7f1354109b70 2024-05-27T08:27:33.991 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.480+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f13541088c0 con 0x7f134003e4c0 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.481+0000 7f13588f0700 1 -- 172.21.15.77:0/1666851203 <== mgr.14120 v2:172.21.15.77:6800/2456305222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f13541088c0 con 0x7f134003e4c0 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f134003e4c0 msgr2=0x7f1340040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 --2- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f134003e4c0 0x7f1340040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f133c006fd0 tx=0x7f133c006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 msgr2=0x7f13541093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 --2- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f13541093f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1344011ac0 tx=0x7f1344011ba0 comp rx=0 tx=0).stop 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 shutdown_connections 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 --2- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f134003e4c0 0x7f1340040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 --2- 172.21.15.77:0/1666851203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1354109b70 0x7f13541093f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 >> 172.21.15.77:0/1666851203 conn(0x7f1354103540 msgr2=0x7f13541040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 shutdown_connections 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:33.483+0000 7f13598f2700 1 -- 172.21.15.77:0/1666851203 wait complete. 2024-05-27T08:27:33.992 INFO:teuthology.orchestra.run.smithi077.stdout:Generating ssh key... 2024-05-27T08:27:34.306 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:34 smithi077 ceph-mon[32007]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:27:34.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: Generating public/private rsa key pair. 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: Your identification has been saved in /tmp/tmpfltd5l8c/key. 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: Your public key has been saved in /tmp/tmpfltd5l8c/key.pub. 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: The key fingerprint is: 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: SHA256:c1Mhc3aPDKVxT2IO9RQmP5naHd7zNG6m33urkhyzXrY ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: The key's randomart image is: 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: +---[RSA 3072]----+ 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | o B+Bo+| 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | = %+Oo| 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | o +*+| 2024-05-27T08:27:34.880 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | . + =| 2024-05-27T08:27:34.881 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | S o . =+| 2024-05-27T08:27:34.881 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | o + ..+| 2024-05-27T08:27:34.881 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | . =o +.| 2024-05-27T08:27:34.881 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | =o = o| 2024-05-27T08:27:34.881 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: | ...Eo+=| 2024-05-27T08:27:34.881 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: +----[SHA256]-----+ 2024-05-27T08:27:35.376 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.404+0000 7f1a1d3da700 1 Processor -- start 2024-05-27T08:27:35.376 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.404+0000 7f1a1d3da700 1 -- start start 2024-05-27T08:27:35.376 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.404+0000 7f1a1d3da700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1810bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.404+0000 7f1a1d3da700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1807ce40 con 0x7f1a18109b60 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.404+0000 7f1a16ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1810bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.404+0000 7f1a16ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1810bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55318/0 (socket says 172.21.15.77:55318) 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.404+0000 7f1a16ffd700 1 -- 172.21.15.77:0/4012560461 learned_addr learned my addr 172.21.15.77:0/4012560461 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.405+0000 7f1a16ffd700 1 -- 172.21.15.77:0/4012560461 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1810c490 con 0x7f1a18109b60 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.405+0000 7f1a16ffd700 1 --2- 172.21.15.77:0/4012560461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1810bf50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a080098a0 tx=0x7f1a08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e47ea06ed91f49 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.405+0000 7f1a15ffb700 1 -- 172.21.15.77:0/4012560461 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a08004160 con 0x7f1a18109b60 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.405+0000 7f1a15ffb700 1 -- 172.21.15.77:0/4012560461 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1a08004300 con 0x7f1a18109b60 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.405+0000 7f1a15ffb700 1 -- 172.21.15.77:0/4012560461 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a08011af0 con 0x7f1a18109b60 2024-05-27T08:27:35.377 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.405+0000 7f1a1d3da700 1 -- 172.21.15.77:0/4012560461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 msgr2=0x7f1a1810bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.405+0000 7f1a1d3da700 1 --2- 172.21.15.77:0/4012560461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1810bf50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a080098a0 tx=0x7f1a08009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.406+0000 7f1a1d3da700 1 -- 172.21.15.77:0/4012560461 shutdown_connections 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.406+0000 7f1a1d3da700 1 --2- 172.21.15.77:0/4012560461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1810bf50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.406+0000 7f1a1d3da700 1 -- 172.21.15.77:0/4012560461 >> 172.21.15.77:0/4012560461 conn(0x7f1a18103570 msgr2=0x7f1a18105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.406+0000 7f1a1d3da700 1 -- 172.21.15.77:0/4012560461 shutdown_connections 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.406+0000 7f1a1d3da700 1 -- 172.21.15.77:0/4012560461 wait complete. 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.406+0000 7f1a1d3da700 1 Processor -- start 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.406+0000 7f1a1d3da700 1 -- start start 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a1d3da700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:35.378 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a1d3da700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1810d400 con 0x7f1a18109b60 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a16ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a16ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55320/0 (socket says 172.21.15.77:55320) 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a16ffd700 1 -- 172.21.15.77:0/869522232 learned_addr learned my addr 172.21.15.77:0/869522232 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a16ffd700 1 -- 172.21.15.77:0/869522232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1819d520 con 0x7f1a18109b60 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a16ffd700 1 --2- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1819cfe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a08011ac0 tx=0x7f1a08011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f19fffff700 1 -- 172.21.15.77:0/869522232 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a08003970 con 0x7f1a18109b60 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f19fffff700 1 -- 172.21.15.77:0/869522232 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1a08003b10 con 0x7f1a18109b60 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1819d7b0 con 0x7f1a18109b60 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f19fffff700 1 -- 172.21.15.77:0/869522232 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a08024610 con 0x7f1a18109b60 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.407+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a181a04a0 con 0x7f1a18109b60 2024-05-27T08:27:35.379 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.408+0000 7f19fffff700 1 -- 172.21.15.77:0/869522232 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1a08024890 con 0x7f1a18109b60 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.408+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a18064ba0 con 0x7f1a18109b60 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.408+0000 7f19fffff700 1 --2- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f1a0003e470 0x7f1a00040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.408+0000 7f19fffff700 1 -- 172.21.15.77:0/869522232 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a08053a30 con 0x7f1a18109b60 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.409+0000 7f1a167fc700 1 --2- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f1a0003e470 0x7f1a00040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.409+0000 7f1a167fc700 1 --2- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f1a0003e470 0x7f1a00040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c006fd0 tx=0x7f1a0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.414+0000 7f19fffff700 1 -- 172.21.15.77:0/869522232 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a0800a0d0 con 0x7f1a18109b60 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.622+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1a181047f0 con 0x7f1a0003e470 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.811+0000 7f19fffff700 1 -- 172.21.15.77:0/869522232 <== mgr.14120 v2:172.21.15.77:6800/2456305222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1a181047f0 con 0x7f1a0003e470 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f1a0003e470 msgr2=0x7f1a00040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 --2- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f1a0003e470 0x7f1a00040930 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c006fd0 tx=0x7f1a0c006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:35.380 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 msgr2=0x7f1a1819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 --2- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1819cfe0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a08011ac0 tx=0x7f1a08011ba0 comp rx=0 tx=0).stop 2024-05-27T08:27:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 shutdown_connections 2024-05-27T08:27:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 --2- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f1a0003e470 0x7f1a00040930 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 --2- 172.21.15.77:0/869522232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a18109b60 0x7f1a1819cfe0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 >> 172.21.15.77:0/869522232 conn(0x7f1a18103570 msgr2=0x7f1a181040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 shutdown_connections 2024-05-27T08:27:35.381 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:34.813+0000 7f1a1d3da700 1 -- 172.21.15.77:0/869522232 wait complete. 2024-05-27T08:27:35.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:35 smithi077 ceph-mon[32007]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:27:35.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:35 smithi077 ceph-mon[32007]: Generating ssh key... 2024-05-27T08:27:35.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:35 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:35.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:35 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB5Zx2ShVGx1YZnzn14ArZ9mUpWbJHHr5j+DQpjGKBPJ5CzAbe94z+PkSHddPDmQnp9jkDglTfIdCTP9+Jh/up7IkfTDgMmQ3Vp7ZsQ8blAUs9ayGZ38vxqPZyR1HqFqEY+DFTW5S8QsLSMRsY0ov7zAVrpRh2utYUV+CJXVbGINa3mlePhgyycVcls74jSmD78lLbAF01c9bKfOPZi3JANV9xFjbPyoyJ5XQRQUt6Gy4/9D8HFwMxVI0GJiuz7Iz3bKTWyy3lDYfMaYDKObnGbZCz/6OHo0+BZaF9OgmE0jZJeTyJBqkdnirO3iG1VxTriOrwb9zOuQbe26KtAiJ5Ti8wqb3eaVQ9BTt5XpDwnnGdmI+56dVqnqGj5rVQm8joDSIqSmGvfNhgWlcL8ztLzrDKdy7ZEIMqW3BEQujd5MBDpVpf/BmNLUvft/LEDHfheYavNb/EptMIXk4gxOlzRnAuz+Cux19jELrNCxbYxvwzihLRnNjhADMja9i3zk= ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.761+0000 7f53d80bf700 1 Processor -- start 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.761+0000 7f53d80bf700 1 -- start start 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.761+0000 7f53d80bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d01094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.761+0000 7f53d80bf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53d01099e0 con 0x7f53d0109080 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.762+0000 7f53d5e5b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d01094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.762+0000 7f53d5e5b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d01094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55326/0 (socket says 172.21.15.77:55326) 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.762+0000 7f53d5e5b700 1 -- 172.21.15.77:0/3374281536 learned_addr learned my addr 172.21.15.77:0/3374281536 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.762+0000 7f53d5e5b700 1 -- 172.21.15.77:0/3374281536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d0109b60 con 0x7f53d0109080 2024-05-27T08:27:36.566 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.762+0000 7f53d5e5b700 1 --2- 172.21.15.77:0/3374281536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d01094a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0098a0 tx=0x7f53cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd84d704627b5bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.762+0000 7f53d4e59700 1 -- 172.21.15.77:0/3374281536 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53cc004160 con 0x7f53d0109080 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.762+0000 7f53d4e59700 1 -- 172.21.15.77:0/3374281536 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53cc004300 con 0x7f53d0109080 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.762+0000 7f53d4e59700 1 -- 172.21.15.77:0/3374281536 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53cc011af0 con 0x7f53d0109080 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.763+0000 7f53d80bf700 1 -- 172.21.15.77:0/3374281536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 msgr2=0x7f53d01094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.763+0000 7f53d80bf700 1 --2- 172.21.15.77:0/3374281536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d01094a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0098a0 tx=0x7f53cc009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.763+0000 7f53d80bf700 1 -- 172.21.15.77:0/3374281536 shutdown_connections 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.763+0000 7f53d80bf700 1 --2- 172.21.15.77:0/3374281536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d01094a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.763+0000 7f53d80bf700 1 -- 172.21.15.77:0/3374281536 >> 172.21.15.77:0/3374281536 conn(0x7f53d0104720 msgr2=0x7f53d0106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.763+0000 7f53d80bf700 1 -- 172.21.15.77:0/3374281536 shutdown_connections 2024-05-27T08:27:36.567 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.763+0000 7f53d80bf700 1 -- 172.21.15.77:0/3374281536 wait complete. 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.763+0000 7f53d80bf700 1 Processor -- start 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d80bf700 1 -- start start 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d80bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d80bf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f53d010b080 con 0x7f53d0109080 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d5e5b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d5e5b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55340/0 (socket says 172.21.15.77:55340) 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d5e5b700 1 -- 172.21.15.77:0/3548501290 learned_addr learned my addr 172.21.15.77:0/3548501290 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d5e5b700 1 -- 172.21.15.77:0/3548501290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d019d4a0 con 0x7f53d0109080 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d5e5b700 1 --2- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d019cf60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f53cc011ac0 tx=0x7f53cc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53c6ffd700 1 -- 172.21.15.77:0/3548501290 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53cc003b20 con 0x7f53d0109080 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53c6ffd700 1 -- 172.21.15.77:0/3548501290 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53cc01a8c0 con 0x7f53d0109080 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53c6ffd700 1 -- 172.21.15.77:0/3548501290 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53cc0248d0 con 0x7f53d0109080 2024-05-27T08:27:36.568 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d019d730 con 0x7f53d0109080 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.764+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d01a0420 con 0x7f53d0109080 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.765+0000 7f53c6ffd700 1 -- 172.21.15.77:0/3548501290 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f53cc022020 con 0x7f53d0109080 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.765+0000 7f53c6ffd700 1 --2- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f53bc03e4c0 0x7f53bc040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.765+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d0064ba0 con 0x7f53d0109080 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.766+0000 7f53c6ffd700 1 -- 172.21.15.77:0/3548501290 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f53cc053830 con 0x7f53d0109080 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.766+0000 7f53d565a700 1 --2- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f53bc03e4c0 0x7f53bc040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.766+0000 7f53d565a700 1 --2- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f53bc03e4c0 0x7f53bc040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006fd0 tx=0x7f53c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.771+0000 7f53c6ffd700 1 -- 172.21.15.77:0/3548501290 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53cc00e260 con 0x7f53d0109080 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.975+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f53d0105920 con 0x7f53bc03e4c0 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.976+0000 7f53c6ffd700 1 -- 172.21.15.77:0/3548501290 <== mgr.14120 v2:172.21.15.77:6800/2456305222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f53d0105920 con 0x7f53bc03e4c0 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f53bc03e4c0 msgr2=0x7f53bc040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:36.569 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 --2- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f53bc03e4c0 0x7f53bc040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006fd0 tx=0x7f53c0006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 msgr2=0x7f53d019cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 --2- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d019cf60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f53cc011ac0 tx=0x7f53cc011ba0 comp rx=0 tx=0).stop 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 shutdown_connections 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 --2- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f53bc03e4c0 0x7f53bc040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 --2- 172.21.15.77:0/3548501290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f53d0109080 0x7f53d019cf60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 >> 172.21.15.77:0/3548501290 conn(0x7f53d0104720 msgr2=0x7f53d0105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 shutdown_connections 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:35.978+0000 7f53d80bf700 1 -- 172.21.15.77:0/3548501290 wait complete. 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:Adding key to root@localhost authorized_keys... 2024-05-27T08:27:36.570 INFO:teuthology.orchestra.run.smithi077.stdout:Adding host smithi077... 2024-05-27T08:27:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:36 smithi077 ceph-mon[32007]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:27:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:38 smithi077 ceph-mon[32007]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:27:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:38 smithi077 ceph-mon[32007]: Deploying cephadm binary to smithi077 2024-05-27T08:27:41.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Added host 'smithi077' with addr '172.21.15.77' 2024-05-27T08:27:41.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.944+0000 7f88ef530700 1 Processor -- start 2024-05-27T08:27:41.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.944+0000 7f88ef530700 1 -- start start 2024-05-27T08:27:41.781 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.944+0000 7f88ef530700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e81094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.944+0000 7f88ef530700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8109a20 con 0x7f88e81090c0 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.944+0000 7f88ed2cc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e81094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.944+0000 7f88ed2cc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e81094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55350/0 (socket says 172.21.15.77:55350) 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.944+0000 7f88ed2cc700 1 -- 172.21.15.77:0/3982265917 learned_addr learned my addr 172.21.15.77:0/3982265917 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88ed2cc700 1 -- 172.21.15.77:0/3982265917 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e8109ba0 con 0x7f88e81090c0 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88ed2cc700 1 --2- 172.21.15.77:0/3982265917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e81094e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88e40098a0 tx=0x7f88e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69436765eb66c9b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88dffff700 1 -- 172.21.15.77:0/3982265917 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e4004160 con 0x7f88e81090c0 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88dffff700 1 -- 172.21.15.77:0/3982265917 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f88e4004300 con 0x7f88e81090c0 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88dffff700 1 -- 172.21.15.77:0/3982265917 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e4011af0 con 0x7f88e81090c0 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88ef530700 1 -- 172.21.15.77:0/3982265917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 msgr2=0x7f88e81094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88ef530700 1 --2- 172.21.15.77:0/3982265917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e81094e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f88e40098a0 tx=0x7f88e4009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88ef530700 1 -- 172.21.15.77:0/3982265917 shutdown_connections 2024-05-27T08:27:41.782 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88ef530700 1 --2- 172.21.15.77:0/3982265917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e81094e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.945+0000 7f88ef530700 1 -- 172.21.15.77:0/3982265917 >> 172.21.15.77:0/3982265917 conn(0x7f88e8104720 msgr2=0x7f88e8106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ef530700 1 -- 172.21.15.77:0/3982265917 shutdown_connections 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ef530700 1 -- 172.21.15.77:0/3982265917 wait complete. 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ef530700 1 Processor -- start 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ef530700 1 -- start start 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ef530700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ef530700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810b070 con 0x7f88e81090c0 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ed2cc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ed2cc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e819cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55358/0 (socket says 172.21.15.77:55358) 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.946+0000 7f88ed2cc700 1 -- 172.21.15.77:0/3309519694 learned_addr learned my addr 172.21.15.77:0/3309519694 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.947+0000 7f88ed2cc700 1 -- 172.21.15.77:0/3309519694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e819d490 con 0x7f88e81090c0 2024-05-27T08:27:41.783 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.947+0000 7f88ed2cc700 1 --2- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e819cf50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88e4011ac0 tx=0x7f88e4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.947+0000 7f88de7fc700 1 -- 172.21.15.77:0/3309519694 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e4003b20 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.947+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e819d720 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.947+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e81a0410 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.947+0000 7f88de7fc700 1 -- 172.21.15.77:0/3309519694 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f88e401a8c0 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.947+0000 7f88de7fc700 1 -- 172.21.15.77:0/3309519694 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88e4024870 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.948+0000 7f88de7fc700 1 -- 172.21.15.77:0/3309519694 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f88e4022020 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.948+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88cc005360 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.948+0000 7f88de7fc700 1 --2- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f88d403e470 0x7f88d4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.948+0000 7f88de7fc700 1 -- 172.21.15.77:0/3309519694 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f88e4053960 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.948+0000 7f88ecacb700 1 --2- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f88d403e470 0x7f88d4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.949+0000 7f88ecacb700 1 --2- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f88d403e470 0x7f88d4040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f88d8006fd0 tx=0x7f88d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:36.953+0000 7f88de7fc700 1 -- 172.21.15.77:0/3309519694 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88e4023370 con 0x7f88e81090c0 2024-05-27T08:27:41.784 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:37.159+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "addr": "172.21.15.77", "target": ["mon-mgr", ""]}) -- 0x7f88cc000bf0 con 0x7f88d403e470 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.226+0000 7f88de7fc700 1 -- 172.21.15.77:0/3309519694 <== mgr.14120 v2:172.21.15.77:6800/2456305222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f88cc000bf0 con 0x7f88d403e470 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.229+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f88d403e470 msgr2=0x7f88d4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.229+0000 7f88ef530700 1 --2- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f88d403e470 0x7f88d4040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f88d8006fd0 tx=0x7f88d8006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.229+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 msgr2=0x7f88e819cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.229+0000 7f88ef530700 1 --2- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e819cf50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88e4011ac0 tx=0x7f88e4011ba0 comp rx=0 tx=0).stop 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.229+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 shutdown_connections 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.229+0000 7f88ef530700 1 --2- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f88d403e470 0x7f88d4040930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.229+0000 7f88ef530700 1 --2- 172.21.15.77:0/3309519694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88e81090c0 0x7f88e819cf50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.229+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 >> 172.21.15.77:0/3309519694 conn(0x7f88e8104720 msgr2=0x7f88e8105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.230+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 shutdown_connections 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:41.230+0000 7f88ef530700 1 -- 172.21.15.77:0/3309519694 wait complete. 2024-05-27T08:27:41.785 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying unmanaged mon service... 2024-05-27T08:27:42.517 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:42 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:42.517 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:42 smithi077 ceph-mon[32007]: Added host smithi077 2024-05-27T08:27:42.518 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:42 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:27:43.756 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:43 smithi077 ceph-mon[32007]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:27:43.756 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:43 smithi077 ceph-mon[32007]: Saving service mon spec with placement count:5 2024-05-27T08:27:43.756 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:43 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:44.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-27T08:27:44.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.273+0000 7fe312b61700 1 Processor -- start 2024-05-27T08:27:44.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.273+0000 7fe312b61700 1 -- start start 2024-05-27T08:27:44.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.273+0000 7fe312b61700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:44.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.273+0000 7fe312b61700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c1099b0 con 0x7fe30c109050 2024-05-27T08:27:44.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.273+0000 7fe3108fd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:44.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.273+0000 7fe3108fd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38254/0 (socket says 172.21.15.77:38254) 2024-05-27T08:27:44.022 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.273+0000 7fe3108fd700 1 -- 172.21.15.77:0/2134133296 learned_addr learned my addr 172.21.15.77:0/2134133296 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe3108fd700 1 -- 172.21.15.77:0/2134133296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c109b30 con 0x7fe30c109050 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe3108fd700 1 --2- 172.21.15.77:0/2134133296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c109470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe3000098a0 tx=0x7fe300009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc377110c6e7c0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe30b7fe700 1 -- 172.21.15.77:0/2134133296 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe300004160 con 0x7fe30c109050 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe30b7fe700 1 -- 172.21.15.77:0/2134133296 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe300004300 con 0x7fe30c109050 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe30b7fe700 1 -- 172.21.15.77:0/2134133296 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe300011af0 con 0x7fe30c109050 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe312b61700 1 -- 172.21.15.77:0/2134133296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 msgr2=0x7fe30c109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe312b61700 1 --2- 172.21.15.77:0/2134133296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c109470 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe3000098a0 tx=0x7fe300009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe312b61700 1 -- 172.21.15.77:0/2134133296 shutdown_connections 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe312b61700 1 --2- 172.21.15.77:0/2134133296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c109470 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.274+0000 7fe312b61700 1 -- 172.21.15.77:0/2134133296 >> 172.21.15.77:0/2134133296 conn(0x7fe30c104670 msgr2=0x7fe30c106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe312b61700 1 -- 172.21.15.77:0/2134133296 shutdown_connections 2024-05-27T08:27:44.023 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe312b61700 1 -- 172.21.15.77:0/2134133296 wait complete. 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe312b61700 1 Processor -- start 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe312b61700 1 -- start start 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe312b61700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe312b61700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c10b050 con 0x7fe30c109050 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe3108fd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe3108fd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c19cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38256/0 (socket says 172.21.15.77:38256) 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.275+0000 7fe3108fd700 1 -- 172.21.15.77:0/838442383 learned_addr learned my addr 172.21.15.77:0/838442383 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.276+0000 7fe3108fd700 1 -- 172.21.15.77:0/838442383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c19d4f0 con 0x7fe30c109050 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.276+0000 7fe3108fd700 1 --2- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c19cfb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe300009fb0 tx=0x7fe300011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.276+0000 7fe309ffb700 1 -- 172.21.15.77:0/838442383 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3000036a0 con 0x7fe30c109050 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.276+0000 7fe309ffb700 1 -- 172.21.15.77:0/838442383 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe30001a430 con 0x7fe30c109050 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.276+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30c19d780 con 0x7fe30c109050 2024-05-27T08:27:44.024 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.276+0000 7fe309ffb700 1 -- 172.21.15.77:0/838442383 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe300019550 con 0x7fe30c109050 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.276+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe30c1a0470 con 0x7fe30c109050 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.277+0000 7fe309ffb700 1 -- 172.21.15.77:0/838442383 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe30001a8e0 con 0x7fe30c109050 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.277+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30c064ba0 con 0x7fe30c109050 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.277+0000 7fe309ffb700 1 --2- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe2fc03e470 0x7fe2fc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.277+0000 7fe309ffb700 1 -- 172.21.15.77:0/838442383 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe300053920 con 0x7fe30c109050 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.277+0000 7fe30bfff700 1 --2- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe2fc03e470 0x7fe2fc040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.278+0000 7fe30bfff700 1 --2- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe2fc03e470 0x7fe2fc040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006fd0 tx=0x7fe2f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.282+0000 7fe309ffb700 1 -- 172.21.15.77:0/838442383 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe30000b3e0 con 0x7fe30c109050 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.515+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe30c1058e0 con 0x7fe2fc03e470 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.523+0000 7fe309ffb700 1 -- 172.21.15.77:0/838442383 <== mgr.14120 v2:172.21.15.77:6800/2456305222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe30c1058e0 con 0x7fe2fc03e470 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe2fc03e470 msgr2=0x7fe2fc040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:44.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 --2- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe2fc03e470 0x7fe2fc040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8006fd0 tx=0x7fe2f8006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 msgr2=0x7fe30c19cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 --2- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c19cfb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe300009fb0 tx=0x7fe300011940 comp rx=0 tx=0).stop 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 shutdown_connections 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 --2- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe2fc03e470 0x7fe2fc040930 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 --2- 172.21.15.77:0/838442383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe30c109050 0x7fe30c19cfb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 >> 172.21.15.77:0/838442383 conn(0x7fe30c104670 msgr2=0x7fe30c1051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 shutdown_connections 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:42.525+0000 7fe312b61700 1 -- 172.21.15.77:0/838442383 wait complete. 2024-05-27T08:27:44.026 INFO:teuthology.orchestra.run.smithi077.stdout:Deploying unmanaged mgr service... 2024-05-27T08:27:44.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:44 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:44.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:44 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:45.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:45 smithi077 ceph-mon[32007]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:27:45.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:45 smithi077 ceph-mon[32007]: Saving service mgr spec with placement count:2 2024-05-27T08:27:45.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:45 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:45.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:45 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:45.948 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-27T08:27:45.948 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.518+0000 7f4144293700 1 Processor -- start 2024-05-27T08:27:45.948 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.519+0000 7f4144293700 1 -- start start 2024-05-27T08:27:45.948 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.519+0000 7f4144293700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c109a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.519+0000 7f4144293700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f413c107b90 con 0x7f413c109650 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.519+0000 7f414202f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c109a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.519+0000 7f414202f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c109a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38262/0 (socket says 172.21.15.77:38262) 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.519+0000 7f414202f700 1 -- 172.21.15.77:0/2978818040 learned_addr learned my addr 172.21.15.77:0/2978818040 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.519+0000 7f414202f700 1 -- 172.21.15.77:0/2978818040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c107d10 con 0x7f413c109650 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.520+0000 7f414202f700 1 --2- 172.21.15.77:0/2978818040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c109a70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f41380098a0 tx=0x7f4138009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ff3d068e0344c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.520+0000 7f414102d700 1 -- 172.21.15.77:0/2978818040 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4138004160 con 0x7f413c109650 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.520+0000 7f414102d700 1 -- 172.21.15.77:0/2978818040 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4138004300 con 0x7f413c109650 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.520+0000 7f414102d700 1 -- 172.21.15.77:0/2978818040 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4138011a20 con 0x7f413c109650 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.520+0000 7f4144293700 1 -- 172.21.15.77:0/2978818040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 msgr2=0x7f413c109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.520+0000 7f4144293700 1 --2- 172.21.15.77:0/2978818040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c109a70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f41380098a0 tx=0x7f4138009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.521+0000 7f4144293700 1 -- 172.21.15.77:0/2978818040 shutdown_connections 2024-05-27T08:27:45.949 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.521+0000 7f4144293700 1 --2- 172.21.15.77:0/2978818040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c109a70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.521+0000 7f4144293700 1 -- 172.21.15.77:0/2978818040 >> 172.21.15.77:0/2978818040 conn(0x7f413c103510 msgr2=0x7f413c105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.521+0000 7f4144293700 1 -- 172.21.15.77:0/2978818040 shutdown_connections 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.521+0000 7f4144293700 1 -- 172.21.15.77:0/2978818040 wait complete. 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.521+0000 7f4144293700 1 Processor -- start 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.521+0000 7f4144293700 1 -- start start 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f4144293700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f4144293700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f413c10c520 con 0x7f413c109650 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f414202f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f414202f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38276/0 (socket says 172.21.15.77:38276) 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f414202f700 1 -- 172.21.15.77:0/2879658419 learned_addr learned my addr 172.21.15.77:0/2879658419 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f414202f700 1 -- 172.21.15.77:0/2879658419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c19c4d0 con 0x7f413c109650 2024-05-27T08:27:45.950 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f414202f700 1 --2- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c19bf90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4138009520 tx=0x7f4138011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f41337fe700 1 -- 172.21.15.77:0/2879658419 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41380039e0 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f41337fe700 1 -- 172.21.15.77:0/2879658419 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f413801aa10 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f41337fe700 1 -- 172.21.15.77:0/2879658419 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4138023990 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c19c760 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.522+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c07bfc0 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.523+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413c10c6a0 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.524+0000 7f41337fe700 1 -- 172.21.15.77:0/2879658419 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4138023c00 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.524+0000 7f41337fe700 1 --2- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f412803e4c0 0x7f4128040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.524+0000 7f41337fe700 1 -- 172.21.15.77:0/2879658419 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4138055500 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.524+0000 7f414182e700 1 --2- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f412803e4c0 0x7f4128040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.524+0000 7f414182e700 1 --2- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f412803e4c0 0x7f4128040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f412c006fd0 tx=0x7f412c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.533+0000 7f41337fe700 1 -- 172.21.15.77:0/2879658419 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4138020290 con 0x7f413c109650 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.773+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f413c104760 con 0x7f412803e4c0 2024-05-27T08:27:45.951 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.780+0000 7f41337fe700 1 -- 172.21.15.77:0/2879658419 <== mgr.14120 v2:172.21.15.77:6800/2456305222 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f413c104760 con 0x7f412803e4c0 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f412803e4c0 msgr2=0x7f4128040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 --2- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f412803e4c0 0x7f4128040980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f412c006fd0 tx=0x7f412c006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 msgr2=0x7f413c19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 --2- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c19bf90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4138009520 tx=0x7f4138011aa0 comp rx=0 tx=0).stop 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 shutdown_connections 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 --2- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f412803e4c0 0x7f4128040980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 --2- 172.21.15.77:0/2879658419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f413c109650 0x7f413c19bf90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 >> 172.21.15.77:0/2879658419 conn(0x7f413c103510 msgr2=0x7f413c104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 shutdown_connections 2024-05-27T08:27:45.952 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:44.783+0000 7f4144293700 1 -- 172.21.15.77:0/2879658419 wait complete. 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.402+0000 7f04911c0700 1 Processor -- start 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.402+0000 7f04911c0700 1 -- start start 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.402+0000 7f04911c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c113350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.402+0000 7f04911c0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f048c113890 con 0x7f048c110f60 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.402+0000 7f048ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c113350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.402+0000 7f048ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c113350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38282/0 (socket says 172.21.15.77:38282) 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.402+0000 7f048ad9d700 1 -- 172.21.15.77:0/4029774688 learned_addr learned my addr 172.21.15.77:0/4029774688 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.403+0000 7f048ad9d700 1 -- 172.21.15.77:0/4029774688 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c113a10 con 0x7f048c110f60 2024-05-27T08:27:47.238 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.403+0000 7f048ad9d700 1 --2- 172.21.15.77:0/4029774688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c113350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f047c010b60 tx=0x7f047c010e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b575828395ca351 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.403+0000 7f0489d9b700 1 -- 172.21.15.77:0/4029774688 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f047c00bdd0 con 0x7f048c110f60 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.403+0000 7f0489d9b700 1 -- 172.21.15.77:0/4029774688 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c0044e0 con 0x7f048c110f60 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.403+0000 7f0489d9b700 1 -- 172.21.15.77:0/4029774688 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f047c0185b0 con 0x7f048c110f60 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.404+0000 7f04911c0700 1 -- 172.21.15.77:0/4029774688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 msgr2=0x7f048c113350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.404+0000 7f04911c0700 1 --2- 172.21.15.77:0/4029774688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c113350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f047c010b60 tx=0x7f047c010e70 comp rx=0 tx=0).stop 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.404+0000 7f04911c0700 1 -- 172.21.15.77:0/4029774688 shutdown_connections 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.404+0000 7f04911c0700 1 --2- 172.21.15.77:0/4029774688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c113350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.404+0000 7f04911c0700 1 -- 172.21.15.77:0/4029774688 >> 172.21.15.77:0/4029774688 conn(0x7f048c06fb80 msgr2=0x7f048c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.404+0000 7f04911c0700 1 -- 172.21.15.77:0/4029774688 shutdown_connections 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.404+0000 7f04911c0700 1 -- 172.21.15.77:0/4029774688 wait complete. 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.405+0000 7f04911c0700 1 Processor -- start 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.405+0000 7f04911c0700 1 -- start start 2024-05-27T08:27:47.239 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f04911c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c1a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f04911c0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f048c075430 con 0x7f048c110f60 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f048ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c1a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f048ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c1a35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38294/0 (socket says 172.21.15.77:38294) 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f048ad9d700 1 -- 172.21.15.77:0/976647484 learned_addr learned my addr 172.21.15.77:0/976647484 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f048ad9d700 1 -- 172.21.15.77:0/976647484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c1a3ae0 con 0x7f048c110f60 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f048ad9d700 1 --2- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c1a35a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f047c010b30 tx=0x7f047c003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f0473fff700 1 -- 172.21.15.77:0/976647484 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f047c026070 con 0x7f048c110f60 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.406+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c1a3d70 con 0x7f048c110f60 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.407+0000 7f0473fff700 1 -- 172.21.15.77:0/976647484 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c018d10 con 0x7f048c110f60 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.407+0000 7f0473fff700 1 -- 172.21.15.77:0/976647484 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f047c021df0 con 0x7f048c110f60 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.407+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c1a41d0 con 0x7f048c110f60 2024-05-27T08:27:47.240 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.407+0000 7f0473fff700 1 -- 172.21.15.77:0/976647484 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f047c029070 con 0x7f048c110f60 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.408+0000 7f0473fff700 1 --2- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f047403e470 0x7f0474040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.408+0000 7f0473fff700 1 -- 172.21.15.77:0/976647484 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f047c058240 con 0x7f048c110f60 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.408+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c064ba0 con 0x7f048c110f60 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.408+0000 7f048a59c700 1 --2- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f047403e470 0x7f0474040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.408+0000 7f048a59c700 1 --2- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f047403e470 0x7f0474040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0480006fd0 tx=0x7f0480006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.414+0000 7f0473fff700 1 -- 172.21.15.77:0/976647484 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f047c01d230 con 0x7f048c110f60 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.626+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f048c1a6f50 con 0x7f048c110f60 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.629+0000 7f0473fff700 1 -- 172.21.15.77:0/976647484 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f047c01cdf0 con 0x7f048c110f60 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f047403e470 msgr2=0x7f0474040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 --2- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f047403e470 0x7f0474040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0480006fd0 tx=0x7f0480006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 msgr2=0x7f048c1a35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 --2- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c1a35a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f047c010b30 tx=0x7f047c003ac0 comp rx=0 tx=0).stop 2024-05-27T08:27:47.241 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 shutdown_connections 2024-05-27T08:27:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 --2- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f047403e470 0x7f0474040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 --2- 172.21.15.77:0/976647484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f048c110f60 0x7f048c1a35a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 >> 172.21.15.77:0/976647484 conn(0x7f048c06fb80 msgr2=0x7f048c071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 shutdown_connections 2024-05-27T08:27:47.242 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:46.633+0000 7f04911c0700 1 -- 172.21.15.77:0/976647484 wait complete. 2024-05-27T08:27:47.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:47 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/976647484' entity='client.admin' 2024-05-27T08:27:47.843 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:47 smithi077 ceph-mon[32007]: from='mgr.14120 172.21.15.77:0/3678892281' entity='mgr.x' 2024-05-27T08:27:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.677+0000 7f0f5e529700 1 Processor -- start 2024-05-27T08:27:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.677+0000 7f0f5e529700 1 -- start start 2024-05-27T08:27:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.677+0000 7f0f5e529700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5810bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.677+0000 7f0f5e529700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5807ce60 con 0x7f0f58109b20 2024-05-27T08:27:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.677+0000 7f0f57fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5810bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.677+0000 7f0f57fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5810bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38300/0 (socket says 172.21.15.77:38300) 2024-05-27T08:27:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.677+0000 7f0f57fff700 1 -- 172.21.15.77:0/2837596928 learned_addr learned my addr 172.21.15.77:0/2837596928 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:48.451 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.677+0000 7f0f57fff700 1 -- 172.21.15.77:0/2837596928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5810c450 con 0x7f0f58109b20 2024-05-27T08:27:48.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.678+0000 7f0f57fff700 1 --2- 172.21.15.77:0/2837596928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5810bf10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0f480098a0 tx=0x7f0f48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8666b71b6bb08f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:48.452 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.678+0000 7f0f577fe700 1 -- 172.21.15.77:0/2837596928 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f48004160 con 0x7f0f58109b20 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.678+0000 7f0f577fe700 1 -- 172.21.15.77:0/2837596928 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f48004300 con 0x7f0f58109b20 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.678+0000 7f0f577fe700 1 -- 172.21.15.77:0/2837596928 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f48011a20 con 0x7f0f58109b20 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.678+0000 7f0f5e529700 1 -- 172.21.15.77:0/2837596928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 msgr2=0x7f0f5810bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.678+0000 7f0f5e529700 1 --2- 172.21.15.77:0/2837596928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5810bf10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0f480098a0 tx=0x7f0f48009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.678+0000 7f0f5e529700 1 -- 172.21.15.77:0/2837596928 shutdown_connections 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.678+0000 7f0f5e529700 1 --2- 172.21.15.77:0/2837596928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5810bf10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f5e529700 1 -- 172.21.15.77:0/2837596928 >> 172.21.15.77:0/2837596928 conn(0x7f0f58103510 msgr2=0x7f0f58105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f5e529700 1 -- 172.21.15.77:0/2837596928 shutdown_connections 2024-05-27T08:27:48.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f5e529700 1 -- 172.21.15.77:0/2837596928 wait complete. 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f5e529700 1 Processor -- start 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f5e529700 1 -- start start 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f5e529700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f5e529700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5810d450 con 0x7f0f58109b20 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f57fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f57fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38314/0 (socket says 172.21.15.77:38314) 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.679+0000 7f0f57fff700 1 -- 172.21.15.77:0/2404864955 learned_addr learned my addr 172.21.15.77:0/2404864955 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.680+0000 7f0f57fff700 1 -- 172.21.15.77:0/2404864955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5819d500 con 0x7f0f58109b20 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.680+0000 7f0f57fff700 1 --2- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5819cfc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f480116e0 tx=0x7f0f48011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.680+0000 7f0f55ffb700 1 -- 172.21.15.77:0/2404864955 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f48003b20 con 0x7f0f58109b20 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.680+0000 7f0f55ffb700 1 -- 172.21.15.77:0/2404864955 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f4801b8f0 con 0x7f0f58109b20 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.680+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5819d790 con 0x7f0f58109b20 2024-05-27T08:27:48.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.680+0000 7f0f55ffb700 1 -- 172.21.15.77:0/2404864955 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f480238c0 con 0x7f0f58109b20 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.680+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f581a0480 con 0x7f0f58109b20 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.681+0000 7f0f55ffb700 1 -- 172.21.15.77:0/2404864955 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f0f48023a60 con 0x7f0f58109b20 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.681+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f58064ba0 con 0x7f0f58109b20 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.681+0000 7f0f55ffb700 1 --2- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f0f4403e470 0x7f0f44040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.681+0000 7f0f55ffb700 1 -- 172.21.15.77:0/2404864955 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0f48053f60 con 0x7f0f58109b20 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.681+0000 7f0f4ffff700 1 --2- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f0f4403e470 0x7f0f44040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.682+0000 7f0f4ffff700 1 --2- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f0f4403e470 0x7f0f44040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0f40006fd0 tx=0x7f0f40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.686+0000 7f0f55ffb700 1 -- 172.21.15.77:0/2404864955 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f4801f030 con 0x7f0f58109b20 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.906+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0f581a0aa0 con 0x7f0f58109b20 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.908+0000 7f0f55ffb700 1 -- 172.21.15.77:0/2404864955 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0f4800b3e0 con 0x7f0f58109b20 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f0f4403e470 msgr2=0x7f0f44040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 --2- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f0f4403e470 0x7f0f44040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0f40006fd0 tx=0x7f0f40006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:48.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 msgr2=0x7f0f5819cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:48.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 --2- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5819cfc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f480116e0 tx=0x7f0f48011aa0 comp rx=0 tx=0).stop 2024-05-27T08:27:48.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 shutdown_connections 2024-05-27T08:27:48.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 --2- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f0f4403e470 0x7f0f44040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:48.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 --2- 172.21.15.77:0/2404864955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f58109b20 0x7f0f5819cfc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:48.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 >> 172.21.15.77:0/2404864955 conn(0x7f0f58103510 msgr2=0x7f0f58104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:48.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 shutdown_connections 2024-05-27T08:27:48.459 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:47.911+0000 7f0f5e529700 1 -- 172.21.15.77:0/2404864955 wait complete. 2024-05-27T08:27:48.459 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling the dashboard module... 2024-05-27T08:27:49.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:48 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2404864955' entity='client.admin' 2024-05-27T08:27:50.233 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:49 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2737971836' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-27T08:27:50.233 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:49 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: ignoring --setuser ceph since I am not root 2024-05-27T08:27:50.234 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:49 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: ignoring --setgroup ceph since I am not root 2024-05-27T08:27:50.234 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:50 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:50.090+0000 7f72b5ede200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T08:27:50.550 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:50 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:50.277+0000 7f72b5ede200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:27:50.550 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:50 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:50.379+0000 7f72b5ede200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf469fb700 1 Processor -- start 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf469fb700 1 -- start start 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf469fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4010bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf469fb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4007ce40 con 0x7fcf40109b10 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf3ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4010bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf3ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4010bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38330/0 (socket says 172.21.15.77:38330) 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf3ffff700 1 -- 172.21.15.77:0/2048413892 learned_addr learned my addr 172.21.15.77:0/2048413892 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf3ffff700 1 -- 172.21.15.77:0/2048413892 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4010c440 con 0x7fcf40109b10 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.333+0000 7fcf3ffff700 1 --2- 172.21.15.77:0/2048413892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4010bf00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf300098a0 tx=0x7fcf30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686e04527c052ab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:50.558 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf3f7fe700 1 -- 172.21.15.77:0/2048413892 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf30004160 con 0x7fcf40109b10 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf3f7fe700 1 -- 172.21.15.77:0/2048413892 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf30004300 con 0x7fcf40109b10 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf3f7fe700 1 -- 172.21.15.77:0/2048413892 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf30011a20 con 0x7fcf40109b10 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf469fb700 1 -- 172.21.15.77:0/2048413892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 msgr2=0x7fcf4010bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf469fb700 1 --2- 172.21.15.77:0/2048413892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4010bf00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcf300098a0 tx=0x7fcf30009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf469fb700 1 -- 172.21.15.77:0/2048413892 shutdown_connections 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf469fb700 1 --2- 172.21.15.77:0/2048413892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4010bf00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf469fb700 1 -- 172.21.15.77:0/2048413892 >> 172.21.15.77:0/2048413892 conn(0x7fcf401034e0 msgr2=0x7fcf40105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf469fb700 1 -- 172.21.15.77:0/2048413892 shutdown_connections 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.334+0000 7fcf469fb700 1 -- 172.21.15.77:0/2048413892 wait complete. 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.335+0000 7fcf469fb700 1 Processor -- start 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.335+0000 7fcf469fb700 1 -- start start 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.335+0000 7fcf469fb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:50.559 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.335+0000 7fcf469fb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4010d3f0 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.335+0000 7fcf3ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.335+0000 7fcf3ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:38332/0 (socket says 172.21.15.77:38332) 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.335+0000 7fcf3ffff700 1 -- 172.21.15.77:0/2737971836 learned_addr learned my addr 172.21.15.77:0/2737971836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.335+0000 7fcf3ffff700 1 -- 172.21.15.77:0/2737971836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4019d490 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.336+0000 7fcf3ffff700 1 --2- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4019cf50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf30004660 tx=0x7fcf30011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.336+0000 7fcf3dffb700 1 -- 172.21.15.77:0/2737971836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf30003aa0 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.336+0000 7fcf3dffb700 1 -- 172.21.15.77:0/2737971836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3001b8f0 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.336+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4019d720 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.336+0000 7fcf3dffb700 1 -- 172.21.15.77:0/2737971836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf30023900 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.336+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf401a0410 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.337+0000 7fcf3dffb700 1 -- 172.21.15.77:0/2737971836 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fcf3001baa0 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.337+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf40064ba0 con 0x7fcf40109b10 2024-05-27T08:27:50.560 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.337+0000 7fcf3dffb700 1 --2- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fcf2c03e470 0x7fcf2c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.337+0000 7fcf3dffb700 1 -- 172.21.15.77:0/2737971836 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcf30053f00 con 0x7fcf40109b10 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.337+0000 7fcf37fff700 1 --2- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fcf2c03e470 0x7fcf2c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.338+0000 7fcf37fff700 1 --2- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fcf2c03e470 0x7fcf2c040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006fd0 tx=0x7fcf28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.343+0000 7fcf3dffb700 1 -- 172.21.15.77:0/2737971836 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf3001f030 con 0x7fcf40109b10 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.604+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcf401a0a00 con 0x7fcf40109b10 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.911+0000 7fcf3dffb700 1 -- 172.21.15.77:0/2737971836 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fcf300202b0 con 0x7fcf40109b10 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.912+0000 7fcf3dffb700 1 -- 172.21.15.77:0/2737971836 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fcf30023b70 con 0x7fcf40109b10 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fcf2c03e470 msgr2=0x7fcf2c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 --2- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fcf2c03e470 0x7fcf2c040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006fd0 tx=0x7fcf28006e40 comp rx=0 tx=0).stop 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 msgr2=0x7fcf4019cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 --2- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4019cf50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcf30004660 tx=0x7fcf30011aa0 comp rx=0 tx=0).stop 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 shutdown_connections 2024-05-27T08:27:50.561 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 --2- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fcf2c03e470 0x7fcf2c040930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:50.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 --2- 172.21.15.77:0/2737971836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf40109b10 0x7fcf4019cf50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:50.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 >> 172.21.15.77:0/2737971836 conn(0x7fcf401034e0 msgr2=0x7fcf40103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:50.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 shutdown_connections 2024-05-27T08:27:50.562 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:49.914+0000 7fcf469fb700 1 -- 172.21.15.77:0/2737971836 wait complete. 2024-05-27T08:27:50.826 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:50 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:50.691+0000 7f72b5ede200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T08:27:50.826 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:50 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:50.777+0000 7f72b5ede200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T08:27:51.095 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:50 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2737971836' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-27T08:27:51.095 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:50 smithi077 ceph-mon[32007]: mgrmap e9: x(active, since 19s) 2024-05-27T08:27:52.065 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.121+0000 7f4f942d1700 1 Processor -- start 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.121+0000 7f4f942d1700 1 -- start start 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.121+0000 7f4f942d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c10bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.121+0000 7f4f942d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c076e00 con 0x7f4f8c109b10 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.121+0000 7f4f9206d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c10bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:52.066 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.121+0000 7f4f9206d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c10bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49634/0 (socket says 172.21.15.77:49634) 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.121+0000 7f4f9206d700 1 -- 172.21.15.77:0/3725049026 learned_addr learned my addr 172.21.15.77:0/3725049026 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.122+0000 7f4f9206d700 1 -- 172.21.15.77:0/3725049026 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c10c440 con 0x7f4f8c109b10 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.122+0000 7f4f9206d700 1 --2- 172.21.15.77:0/3725049026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c10bf00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f880098a0 tx=0x7f4f88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0e9c3074275f9f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.122+0000 7f4f9106b700 1 -- 172.21.15.77:0/3725049026 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f88004160 con 0x7f4f8c109b10 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.122+0000 7f4f9106b700 1 -- 172.21.15.77:0/3725049026 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f88004300 con 0x7f4f8c109b10 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.122+0000 7f4f9106b700 1 -- 172.21.15.77:0/3725049026 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f88011a20 con 0x7f4f8c109b10 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.123+0000 7f4f942d1700 1 -- 172.21.15.77:0/3725049026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 msgr2=0x7f4f8c10bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.123+0000 7f4f942d1700 1 --2- 172.21.15.77:0/3725049026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c10bf00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f880098a0 tx=0x7f4f88009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.123+0000 7f4f942d1700 1 -- 172.21.15.77:0/3725049026 shutdown_connections 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.123+0000 7f4f942d1700 1 --2- 172.21.15.77:0/3725049026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c10bf00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.123+0000 7f4f942d1700 1 -- 172.21.15.77:0/3725049026 >> 172.21.15.77:0/3725049026 conn(0x7f4f8c1034e0 msgr2=0x7f4f8c105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:52.067 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.123+0000 7f4f942d1700 1 -- 172.21.15.77:0/3725049026 shutdown_connections 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.123+0000 7f4f942d1700 1 -- 172.21.15.77:0/3725049026 wait complete. 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f942d1700 1 Processor -- start 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f942d1700 1 -- start start 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f942d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f942d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c10d440 con 0x7f4f8c109b10 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f9206d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f9206d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49638/0 (socket says 172.21.15.77:49638) 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f9206d700 1 -- 172.21.15.77:0/940988968 learned_addr learned my addr 172.21.15.77:0/940988968 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f9206d700 1 -- 172.21.15.77:0/940988968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c1098d0 con 0x7f4f8c109b10 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f9206d700 1 --2- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c109390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f88004660 tx=0x7f4f88011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.124+0000 7f4f837fe700 1 -- 172.21.15.77:0/940988968 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f88003aa0 con 0x7f4f8c109b10 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.125+0000 7f4f837fe700 1 -- 172.21.15.77:0/940988968 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8801b8f0 con 0x7f4f8c109b10 2024-05-27T08:27:52.068 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.125+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c107ac0 con 0x7f4f8c109b10 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.125+0000 7f4f837fe700 1 -- 172.21.15.77:0/940988968 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f88023900 con 0x7f4f8c109b10 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.125+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c108000 con 0x7f4f8c109b10 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.126+0000 7f4f837fe700 1 -- 172.21.15.77:0/940988968 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4f88023b70 con 0x7f4f8c109b10 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.126+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8c064ba0 con 0x7f4f8c109b10 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.126+0000 7f4f837fe700 1 --2- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f4f7803e520 0x7f4f780409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.126+0000 7f4f837fe700 1 -- 172.21.15.77:0/940988968 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4f88053dc0 con 0x7f4f8c109b10 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.126+0000 7f4f9186c700 1 -- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f4f7803e520 msgr2=0x7f4f780409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2456305222 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.126+0000 7f4f9186c700 1 --2- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f4f7803e520 0x7f4f780409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.131+0000 7f4f837fe700 1 -- 172.21.15.77:0/940988968 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4f8801f030 con 0x7f4f8c109b10 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.327+0000 7f4f9186c700 1 -- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f4f7803e520 msgr2=0x7f4f780409e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2456305222 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.327+0000 7f4f9186c700 1 --2- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f4f7803e520 0x7f4f780409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.434+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4f8c1081d0 con 0x7f4f8c109b10 2024-05-27T08:27:52.069 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.434+0000 7f4f837fe700 1 -- 172.21.15.77:0/940988968 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f4f8801bde0 con 0x7f4f8c109b10 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f4f7803e520 msgr2=0x7f4f780409e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 --2- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f4f7803e520 0x7f4f780409e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 msgr2=0x7f4f8c109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 --2- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c109390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f88004660 tx=0x7f4f88011aa0 comp rx=0 tx=0).stop 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 shutdown_connections 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 --2- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7f4f7803e520 0x7f4f780409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 --2- 172.21.15.77:0/940988968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f8c109b10 0x7f4f8c109390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 >> 172.21.15.77:0/940988968 conn(0x7f4f8c1034e0 msgr2=0x7f4f8c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 shutdown_connections 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:51.436+0000 7f4f942d1700 1 -- 172.21.15.77:0/940988968 wait complete. 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for the mgr to restart... 2024-05-27T08:27:52.070 INFO:teuthology.orchestra.run.smithi077.stdout:Waiting for mgr epoch 9... 2024-05-27T08:27:52.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:51 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/940988968' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-27T08:27:52.129 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:52.009+0000 7f72b5ede200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T08:27:52.476 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:52.186+0000 7f72b5ede200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T08:27:52.476 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:52.362+0000 7f72b5ede200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T08:27:52.754 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:52.659+0000 7f72b5ede200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T08:27:52.754 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:52.753+0000 7f72b5ede200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T08:27:53.102 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:52.958+0000 7f72b5ede200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T08:27:53.379 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:53 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:53.154+0000 7f72b5ede200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:27:54.129 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:53 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:53.762+0000 7f72b5ede200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T08:27:54.130 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:53 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:53.985+0000 7f72b5ede200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T08:27:54.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:54 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:54.414+0000 7f72b5ede200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T08:27:54.879 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:54 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:54.512+0000 7f72b5ede200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T08:27:55.240 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:54 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:54.881+0000 7f72b5ede200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T08:27:55.241 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:55 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:55.239+0000 7f72b5ede200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T08:27:55.504 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:55 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:55.334+0000 7f72b5ede200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T08:27:55.814 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:55 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:55.581+0000 7f72b5ede200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T08:27:55.814 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:55 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:55.716+0000 7f72b5ede200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T08:27:55.814 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:55 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:55.813+0000 7f72b5ede200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T08:27:56.629 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:56 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:56.141+0000 7f72b5ede200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T08:27:56.629 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:56 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:56.234+0000 7f72b5ede200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T08:27:57.117 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:56 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:56.838+0000 7f72b5ede200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T08:27:57.117 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:27:57 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:27:57.062+0000 7f72b5ede200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T08:27:57.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: Active manager daemon x restarted 2024-05-27T08:27:57.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: Activating manager daemon x 2024-05-27T08:27:57.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: osdmap e3: 0 total, 0 up, 0 in 2024-05-27T08:27:57.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: mgrmap e10: x(active, starting, since 0.00410874s) 2024-05-27T08:27:57.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:27:57.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T08:27:57.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T08:27:57.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T08:27:57.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T08:27:57.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: Manager daemon x is now available 2024-05-27T08:27:57.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:27:58.282 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T08:27:58.282 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T08:27:58.282 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:58 smithi077 ceph-mon[32007]: [27/May/2024:08:27:57] ENGINE Bus STARTING 2024-05-27T08:27:58.282 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:58 smithi077 ceph-mon[32007]: [27/May/2024:08:27:57] ENGINE Serving on https://172.21.15.77:7150 2024-05-27T08:27:58.283 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:58 smithi077 ceph-mon[32007]: [27/May/2024:08:27:57] ENGINE Serving on http://172.21.15.77:8765 2024-05-27T08:27:58.283 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:58 smithi077 ceph-mon[32007]: [27/May/2024:08:27:57] ENGINE Bus STARTED 2024-05-27T08:27:58.283 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:58 smithi077 ceph-mon[32007]: mgrmap e11: x(active, since 1.00709s) 2024-05-27T08:27:59.025 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout { 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout } 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.494+0000 7fe15e898700 1 Processor -- start 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.494+0000 7fe15e898700 1 -- start start 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.494+0000 7fe15e898700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe158109520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.494+0000 7fe15e898700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe158109a60 con 0x7fe158109100 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.494+0000 7fe157fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe158109520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.494+0000 7fe157fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe158109520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49642/0 (socket says 172.21.15.77:49642) 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.494+0000 7fe157fff700 1 -- 172.21.15.77:0/680513172 learned_addr learned my addr 172.21.15.77:0/680513172 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:59.026 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.494+0000 7fe157fff700 1 -- 172.21.15.77:0/680513172 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe158109be0 con 0x7fe158109100 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.495+0000 7fe157fff700 1 --2- 172.21.15.77:0/680513172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe158109520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1480098a0 tx=0x7fe148009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818b43a5b5e18ec3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.495+0000 7fe156ffd700 1 -- 172.21.15.77:0/680513172 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe148004160 con 0x7fe158109100 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.495+0000 7fe156ffd700 1 -- 172.21.15.77:0/680513172 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe148004300 con 0x7fe158109100 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.495+0000 7fe156ffd700 1 -- 172.21.15.77:0/680513172 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe148011a20 con 0x7fe158109100 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.495+0000 7fe15e898700 1 -- 172.21.15.77:0/680513172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 msgr2=0x7fe158109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.495+0000 7fe15e898700 1 --2- 172.21.15.77:0/680513172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe158109520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1480098a0 tx=0x7fe148009bb0 comp rx=0 tx=0).stop 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.496+0000 7fe15e898700 1 -- 172.21.15.77:0/680513172 shutdown_connections 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.496+0000 7fe15e898700 1 --2- 172.21.15.77:0/680513172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe158109520 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.496+0000 7fe15e898700 1 -- 172.21.15.77:0/680513172 >> 172.21.15.77:0/680513172 conn(0x7fe158104760 msgr2=0x7fe158106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.496+0000 7fe15e898700 1 -- 172.21.15.77:0/680513172 shutdown_connections 2024-05-27T08:27:59.027 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.496+0000 7fe15e898700 1 -- 172.21.15.77:0/680513172 wait complete. 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.496+0000 7fe15e898700 1 Processor -- start 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.496+0000 7fe15e898700 1 -- start start 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe15e898700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe15819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe15e898700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe15810b140 con 0x7fe158109100 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe157fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe15819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe157fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe15819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49648/0 (socket says 172.21.15.77:49648) 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe157fff700 1 -- 172.21.15.77:0/1689159457 learned_addr learned my addr 172.21.15.77:0/1689159457 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe157fff700 1 -- 172.21.15.77:0/1689159457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15819d460 con 0x7fe158109100 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe157fff700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe15819cf20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe1480116e0 tx=0x7fe148011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:59.028 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe148003a60 con 0x7fe158109100 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15819d6f0 con 0x7fe158109100 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe14801b8f0 con 0x7fe158109100 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe148019860 con 0x7fe158109100 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.497+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a0390 con 0x7fe158109100 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.498+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fe14801b440 con 0x7fe158109100 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.499+0000 7fe1557fa700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 0x7fe140040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.499+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 --> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe1400410a0 con 0x7fe14003e4d0 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.499+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe148053c60 con 0x7fe158109100 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.499+0000 7fe1577fe700 1 -- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 msgr2=0x7fe140040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2456305222 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.499+0000 7fe1577fe700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 0x7fe140040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.700+0000 7fe1577fe700 1 -- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 msgr2=0x7fe140040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2456305222 2024-05-27T08:27:59.029 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:52.700+0000 7fe1577fe700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 0x7fe140040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:53.101+0000 7fe1577fe700 1 -- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 msgr2=0x7fe140040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2456305222 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:53.101+0000 7fe1577fe700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 0x7fe140040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:53.902+0000 7fe1577fe700 1 -- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 msgr2=0x7fe140040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2456305222 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:53.902+0000 7fe1577fe700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 0x7fe140040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:55.503+0000 7fe1577fe700 1 -- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 msgr2=0x7fe140040990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2456305222 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:55.503+0000 7fe1577fe700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 0x7fe140040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:57.066+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fe14800fd60 con 0x7fe158109100 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:57.067+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 msgr2=0x7fe140040990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:57.067+0000 7fe1557fa700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 0x7fe140040990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:59.030 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.068+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fe148055d10 con 0x7fe158109100 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.068+0000 7fe1557fa700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe140041f50 0x7fe140044340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.068+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe148022e70 con 0x7fe140041f50 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.069+0000 7fe1577fe700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe140041f50 0x7fe140044340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.070+0000 7fe1577fe700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe140041f50 0x7fe140044340 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe14c003a10 tx=0x7fe14c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.070+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fe148022e70 con 0x7fe140041f50 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe15806a6a0 con 0x7fe140041f50 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe1557fa700 1 -- 172.21.15.77:0/1689159457 <== mgr.14152 v2:172.21.15.77:6800/2118880698 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fe15806a6a0 con 0x7fe140041f50 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe140041f50 msgr2=0x7fe140044340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe140041f50 0x7fe140044340 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe14c003a10 tx=0x7fe14c0092b0 comp rx=0 tx=0).stop 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 msgr2=0x7fe15819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe15819cf20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe1480116e0 tx=0x7fe148011aa0 comp rx=0 tx=0).stop 2024-05-27T08:27:59.031 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 shutdown_connections 2024-05-27T08:27:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe140041f50 0x7fe140044340 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe158109100 0x7fe15819cf20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 --2- 172.21.15.77:0/1689159457 >> [v2:172.21.15.77:6800/2456305222,v1:172.21.15.77:6801/2456305222] conn(0x7fe14003e4d0 0x7fe140040990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:27:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.079+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 >> 172.21.15.77:0/1689159457 conn(0x7fe158104760 msgr2=0x7fe158105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:27:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.080+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 shutdown_connections 2024-05-27T08:27:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:58.080+0000 7fe15e898700 1 -- 172.21.15.77:0/1689159457 wait complete. 2024-05-27T08:27:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:mgr epoch 9 is available 2024-05-27T08:27:59.032 INFO:teuthology.orchestra.run.smithi077.stdout:Generating a dashboard self-signed certificate... 2024-05-27T08:28:00.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:59 smithi077 ceph-mon[32007]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-27T08:28:00.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:59 smithi077 ceph-mon[32007]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-27T08:28:00.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:27:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.452+0000 7f28d5a52700 1 Processor -- start 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.452+0000 7f28d5a52700 1 -- start start 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28d5a52700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d0109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28d5a52700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0109a40 con 0x7f28d0109050 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d0109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d0109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49724/0 (socket says 172.21.15.77:49724) 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28ceffd700 1 -- 172.21.15.77:0/2476903175 learned_addr learned my addr 172.21.15.77:0/2476903175 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:00.347 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28ceffd700 1 -- 172.21.15.77:0/2476903175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d010a250 con 0x7f28d0109050 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28ceffd700 1 --2- 172.21.15.77:0/2476903175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d0109470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28c00098a0 tx=0x7f28c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27767e590b23346 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28ce7fc700 1 -- 172.21.15.77:0/2476903175 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c0004160 con 0x7f28d0109050 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28ce7fc700 1 -- 172.21.15.77:0/2476903175 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c0004300 con 0x7f28d0109050 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.453+0000 7f28ce7fc700 1 -- 172.21.15.77:0/2476903175 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c0011a20 con 0x7f28d0109050 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.454+0000 7f28d5a52700 1 -- 172.21.15.77:0/2476903175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 msgr2=0x7f28d0109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.454+0000 7f28d5a52700 1 --2- 172.21.15.77:0/2476903175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d0109470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28c00098a0 tx=0x7f28c0009bb0 comp rx=0 tx=0).stop 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.454+0000 7f28d5a52700 1 -- 172.21.15.77:0/2476903175 shutdown_connections 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.454+0000 7f28d5a52700 1 --2- 172.21.15.77:0/2476903175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d0109470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.454+0000 7f28d5a52700 1 -- 172.21.15.77:0/2476903175 >> 172.21.15.77:0/2476903175 conn(0x7f28d0104630 msgr2=0x7f28d0106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.454+0000 7f28d5a52700 1 -- 172.21.15.77:0/2476903175 shutdown_connections 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.454+0000 7f28d5a52700 1 -- 172.21.15.77:0/2476903175 wait complete. 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28d5a52700 1 Processor -- start 2024-05-27T08:28:00.348 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28d5a52700 1 -- start start 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28d5a52700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28d5a52700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28d010d600 con 0x7f28d0109050 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d019cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49730/0 (socket says 172.21.15.77:49730) 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28ceffd700 1 -- 172.21.15.77:0/3273864389 learned_addr learned my addr 172.21.15.77:0/3273864389 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28ceffd700 1 -- 172.21.15.77:0/3273864389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d019d4e0 con 0x7f28d0109050 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28ceffd700 1 --2- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d019cfa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f28c00119f0 tx=0x7f28c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28d4a50700 1 -- 172.21.15.77:0/3273864389 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c00039e0 con 0x7f28d0109050 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28d4a50700 1 -- 172.21.15.77:0/3273864389 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c001b8f0 con 0x7f28d0109050 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28d4a50700 1 -- 172.21.15.77:0/3273864389 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28c00237a0 con 0x7f28d0109050 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.455+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d019d770 con 0x7f28d0109050 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.456+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d01a0460 con 0x7f28d0109050 2024-05-27T08:28:00.349 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.456+0000 7f28d4a50700 1 -- 172.21.15.77:0/3273864389 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f28c001b440 con 0x7f28d0109050 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.456+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d0064ba0 con 0x7f28d0109050 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.457+0000 7f28d4a50700 1 --2- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28bc03e3b0 0x7f28bc040870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.457+0000 7f28d4a50700 1 -- 172.21.15.77:0/3273864389 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f28c0053d80 con 0x7f28d0109050 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.457+0000 7f28c67fc700 1 --2- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28bc03e3b0 0x7f28bc040870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.457+0000 7f28c67fc700 1 --2- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28bc03e3b0 0x7f28bc040870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006fd0 tx=0x7f28b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.462+0000 7f28d4a50700 1 -- 172.21.15.77:0/3273864389 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28c001f030 con 0x7f28d0109050 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.679+0000 7f28d4a50700 1 -- 172.21.15.77:0/3273864389 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f28c005a080 con 0x7f28d0109050 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.684+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f28d0105850 con 0x7f28bc03e3b0 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.765+0000 7f28d4a50700 1 -- 172.21.15.77:0/3273864389 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f28d0105850 con 0x7f28bc03e3b0 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28bc03e3b0 msgr2=0x7f28bc040870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 --2- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28bc03e3b0 0x7f28bc040870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006fd0 tx=0x7f28b8006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:00.350 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 msgr2=0x7f28d019cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:00.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 --2- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d019cfa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f28c00119f0 tx=0x7f28c0011ad0 comp rx=0 tx=0).stop 2024-05-27T08:28:00.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 shutdown_connections 2024-05-27T08:28:00.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 --2- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28bc03e3b0 0x7f28bc040870 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:00.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 --2- 172.21.15.77:0/3273864389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d0109050 0x7f28d019cfa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:00.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 >> 172.21.15.77:0/3273864389 conn(0x7f28d0104630 msgr2=0x7f28d0105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:00.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 shutdown_connections 2024-05-27T08:28:00.351 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:27:59.768+0000 7f28d5a52700 1 -- 172.21.15.77:0/3273864389 wait complete. 2024-05-27T08:28:00.351 INFO:teuthology.orchestra.run.smithi077.stdout:Creating initial admin user... 2024-05-27T08:28:00.792 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:00 smithi077 ceph-mon[32007]: mgrmap e12: x(active, since 2s) 2024-05-27T08:28:00.792 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:00 smithi077 ceph-mon[32007]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:28:00.793 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:00.793 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:00.793 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:02.216 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$wBcSi.DwR/puwsWZvB6X9eSnV/Zjum4dvQ/gzFBnEw.9wz88KAs8y", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1716798481, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.814+0000 7f1a800c8700 1 Processor -- start 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.814+0000 7f1a800c8700 1 -- start start 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.814+0000 7f1a800c8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7810bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.814+0000 7f1a800c8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7807ce40 con 0x7f1a78109b10 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.814+0000 7f1a7de64700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7810bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.814+0000 7f1a7de64700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7810bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46046/0 (socket says 172.21.15.77:46046) 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.814+0000 7f1a7de64700 1 -- 172.21.15.77:0/1118237808 learned_addr learned my addr 172.21.15.77:0/1118237808 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.815+0000 7f1a7de64700 1 -- 172.21.15.77:0/1118237808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7810c440 con 0x7f1a78109b10 2024-05-27T08:28:02.217 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.815+0000 7f1a7de64700 1 --2- 172.21.15.77:0/1118237808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7810bf00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a740098a0 tx=0x7f1a74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766b1c2c0a626a6e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.815+0000 7f1a7ce62700 1 -- 172.21.15.77:0/1118237808 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a74004160 con 0x7f1a78109b10 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.815+0000 7f1a7ce62700 1 -- 172.21.15.77:0/1118237808 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a74004300 con 0x7f1a78109b10 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.815+0000 7f1a7ce62700 1 -- 172.21.15.77:0/1118237808 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a74011a20 con 0x7f1a78109b10 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 -- 172.21.15.77:0/1118237808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 msgr2=0x7f1a7810bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 --2- 172.21.15.77:0/1118237808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7810bf00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a740098a0 tx=0x7f1a74009bb0 comp rx=0 tx=0).stop 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 -- 172.21.15.77:0/1118237808 shutdown_connections 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 --2- 172.21.15.77:0/1118237808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7810bf00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 -- 172.21.15.77:0/1118237808 >> 172.21.15.77:0/1118237808 conn(0x7f1a781034e0 msgr2=0x7f1a78105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 -- 172.21.15.77:0/1118237808 shutdown_connections 2024-05-27T08:28:02.218 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 -- 172.21.15.77:0/1118237808 wait complete. 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 Processor -- start 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.816+0000 7f1a800c8700 1 -- start start 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a800c8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a800c8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7810d3f0 con 0x7f1a78109b10 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a7de64700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a7de64700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46058/0 (socket says 172.21.15.77:46058) 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a7de64700 1 -- 172.21.15.77:0/599702810 learned_addr learned my addr 172.21.15.77:0/599702810 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a7de64700 1 -- 172.21.15.77:0/599702810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7819d4b0 con 0x7f1a78109b10 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a7de64700 1 --2- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7819cf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a740116e0 tx=0x7f1a74011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a6effd700 1 -- 172.21.15.77:0/599702810 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a74003b20 con 0x7f1a78109b10 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a6effd700 1 -- 172.21.15.77:0/599702810 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a7401b8f0 con 0x7f1a78109b10 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a6effd700 1 -- 172.21.15.77:0/599702810 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a740238c0 con 0x7f1a78109b10 2024-05-27T08:28:02.219 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7819d740 con 0x7f1a78109b10 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.817+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a781a0430 con 0x7f1a78109b10 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.818+0000 7f1a6effd700 1 -- 172.21.15.77:0/599702810 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f1a74023b70 con 0x7f1a78109b10 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.818+0000 7f1a6effd700 1 --2- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1a6403e470 0x7f1a64040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.819+0000 7f1a6effd700 1 -- 172.21.15.77:0/599702810 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1a74054510 con 0x7f1a78109b10 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.819+0000 7f1a7d663700 1 --2- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1a6403e470 0x7f1a64040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.819+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a78064ba0 con 0x7f1a78109b10 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.819+0000 7f1a7d663700 1 --2- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1a6403e470 0x7f1a64040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a68006fd0 tx=0x7f1a68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:00.824+0000 7f1a6effd700 1 -- 172.21.15.77:0/599702810 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1a7401f030 con 0x7f1a78109b10 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.063+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1a781a0600 con 0x7f1a6403e470 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.286+0000 7f1a6effd700 1 -- 172.21.15.77:0/599702810 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1a781a0600 con 0x7f1a6403e470 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.288+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1a6403e470 msgr2=0x7f1a64040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.288+0000 7f1a800c8700 1 --2- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1a6403e470 0x7f1a64040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1a68006fd0 tx=0x7f1a68006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:02.220 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.288+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 msgr2=0x7f1a7819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:02.221 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.288+0000 7f1a800c8700 1 --2- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7819cf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1a740116e0 tx=0x7f1a74011aa0 comp rx=0 tx=0).stop 2024-05-27T08:28:02.221 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.288+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 shutdown_connections 2024-05-27T08:28:02.221 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.288+0000 7f1a800c8700 1 --2- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1a6403e470 0x7f1a64040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:02.221 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.288+0000 7f1a800c8700 1 --2- 172.21.15.77:0/599702810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1a78109b10 0x7f1a7819cf70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:02.221 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.288+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 >> 172.21.15.77:0/599702810 conn(0x7f1a781034e0 msgr2=0x7f1a78103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:02.221 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.289+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 shutdown_connections 2024-05-27T08:28:02.221 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:01.289+0000 7f1a800c8700 1 -- 172.21.15.77:0/599702810 wait complete. 2024-05-27T08:28:02.221 INFO:teuthology.orchestra.run.smithi077.stdout:Fetching dashboard port number... 2024-05-27T08:28:02.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:02 smithi077 ceph-mon[32007]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:28:02.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stdout 8443 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.100+0000 7fbe49f84700 1 Processor -- start 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe49f84700 1 -- start start 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe49f84700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4410bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe49f84700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4407ce40 con 0x7fbe44109b60 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe437fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4410bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe437fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4410bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46062/0 (socket says 172.21.15.77:46062) 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe437fe700 1 -- 172.21.15.77:0/3703225604 learned_addr learned my addr 172.21.15.77:0/3703225604 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:03.876 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe437fe700 1 -- 172.21.15.77:0/3703225604 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4410c490 con 0x7fbe44109b60 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe437fe700 1 --2- 172.21.15.77:0/3703225604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4410bf50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009e80 tx=0x7fbe3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe04d108645f9ca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.101+0000 7fbe427fc700 1 -- 172.21.15.77:0/3703225604 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe34017070 con 0x7fbe44109b60 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe427fc700 1 -- 172.21.15.77:0/3703225604 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe3400b8c0 con 0x7fbe44109b60 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe427fc700 1 -- 172.21.15.77:0/3703225604 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe34004490 con 0x7fbe44109b60 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe49f84700 1 -- 172.21.15.77:0/3703225604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 msgr2=0x7fbe4410bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe49f84700 1 --2- 172.21.15.77:0/3703225604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4410bf50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009e80 tx=0x7fbe3400b330 comp rx=0 tx=0).stop 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe49f84700 1 -- 172.21.15.77:0/3703225604 shutdown_connections 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe49f84700 1 --2- 172.21.15.77:0/3703225604 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4410bf50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe49f84700 1 -- 172.21.15.77:0/3703225604 >> 172.21.15.77:0/3703225604 conn(0x7fbe44103570 msgr2=0x7fbe44105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe49f84700 1 -- 172.21.15.77:0/3703225604 shutdown_connections 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.102+0000 7fbe49f84700 1 -- 172.21.15.77:0/3703225604 wait complete. 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.103+0000 7fbe49f84700 1 Processor -- start 2024-05-27T08:28:03.877 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.103+0000 7fbe49f84700 1 -- start start 2024-05-27T08:28:03.878 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.103+0000 7fbe49f84700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:03.878 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.103+0000 7fbe49f84700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4410d440 con 0x7fbe44109b60 2024-05-27T08:28:03.878 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.103+0000 7fbe437fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:03.878 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.103+0000 7fbe437fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4419cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46076/0 (socket says 172.21.15.77:46076) 2024-05-27T08:28:03.878 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.103+0000 7fbe437fe700 1 -- 172.21.15.77:0/1777517535 learned_addr learned my addr 172.21.15.77:0/1777517535 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:03.878 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.103+0000 7fbe437fe700 1 -- 172.21.15.77:0/1777517535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4419d500 con 0x7fbe44109b60 2024-05-27T08:28:03.878 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.104+0000 7fbe437fe700 1 --2- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4419cfc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b7b0 tx=0x7fbe3400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:03.878 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.104+0000 7fbe40ff9700 1 -- 172.21.15.77:0/1777517535 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe34020020 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.104+0000 7fbe40ff9700 1 -- 172.21.15.77:0/1777517535 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe34003c60 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.104+0000 7fbe40ff9700 1 -- 172.21.15.77:0/1777517535 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe34017040 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.104+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4419d790 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.104+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe441a0480 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.105+0000 7fbe40ff9700 1 -- 172.21.15.77:0/1777517535 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fbe34023020 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.105+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe44064ba0 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.105+0000 7fbe40ff9700 1 --2- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbe2c03e4c0 0x7fbe2c040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.105+0000 7fbe40ff9700 1 -- 172.21.15.77:0/1777517535 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe3400f050 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.105+0000 7fbe42ffd700 1 --2- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbe2c03e4c0 0x7fbe2c040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.105+0000 7fbe42ffd700 1 --2- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbe2c03e4c0 0x7fbe2c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe38006fd0 tx=0x7fbe38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.110+0000 7fbe40ff9700 1 -- 172.21.15.77:0/1777517535 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe3402a020 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.287+0000 7fbe40ff9700 1 -- 172.21.15.77:0/1777517535 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbe34055d00 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.319+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fbe441a0650 con 0x7fbe44109b60 2024-05-27T08:28:03.879 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.319+0000 7fbe40ff9700 1 -- 172.21.15.77:0/1777517535 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fbe441a0650 con 0x7fbe44109b60 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbe2c03e4c0 msgr2=0x7fbe2c040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 --2- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbe2c03e4c0 0x7fbe2c040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe38006fd0 tx=0x7fbe38006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 msgr2=0x7fbe4419cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 --2- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4419cfc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe3400b7b0 tx=0x7fbe3400bd50 comp rx=0 tx=0).stop 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 shutdown_connections 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 --2- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbe2c03e4c0 0x7fbe2c040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 --2- 172.21.15.77:0/1777517535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbe44109b60 0x7fbe4419cfc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 >> 172.21.15.77:0/1777517535 conn(0x7fbe44103570 msgr2=0x7fbe44104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.321+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 shutdown_connections 2024-05-27T08:28:03.880 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:03.322+0000 7fbe49f84700 1 -- 172.21.15.77:0/1777517535 wait complete. 2024-05-27T08:28:03.882 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-27T08:28:03.882 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout disabled 2024-05-27T08:28:03.891 INFO:teuthology.orchestra.run.smithi077.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-27T08:28:03.891 INFO:teuthology.orchestra.run.smithi077.stdout:systemctl: stdout inactive 2024-05-27T08:28:03.891 INFO:teuthology.orchestra.run.smithi077.stdout:firewalld.service is not enabled 2024-05-27T08:28:03.891 INFO:teuthology.orchestra.run.smithi077.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-27T08:28:03.892 INFO:teuthology.orchestra.run.smithi077.stdout:Ceph Dashboard is now available at: 2024-05-27T08:28:03.892 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:03.892 INFO:teuthology.orchestra.run.smithi077.stdout: URL: https://smithi077.front.sepia.ceph.com:8443/ 2024-05-27T08:28:03.893 INFO:teuthology.orchestra.run.smithi077.stdout: User: admin 2024-05-27T08:28:03.893 INFO:teuthology.orchestra.run.smithi077.stdout: Password: 4wm5d3lb97 2024-05-27T08:28:03.893 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:03.893 INFO:teuthology.orchestra.run.smithi077.stdout:Saving cluster configuration to /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config directory 2024-05-27T08:28:03.893 INFO:teuthology.orchestra.run.smithi077.stdout:Enabling autotune for osd_memory_target 2024-05-27T08:28:04.501 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:04 smithi077 ceph-mon[32007]: mgrmap e13: x(active, since 6s) 2024-05-27T08:28:04.501 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:04 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1777517535' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-27T08:28:05.072 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.285+0000 7ff03fc91700 1 Processor -- start 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.285+0000 7ff03fc91700 1 -- start start 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.285+0000 7ff03fc91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03807d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.285+0000 7ff03fc91700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff03807d830 con 0x7ff03807ed80 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.285+0000 7ff03da2d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03807d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.285+0000 7ff03da2d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03807d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46084/0 (socket says 172.21.15.77:46084) 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.285+0000 7ff03da2d700 1 -- 172.21.15.77:0/2966079683 learned_addr learned my addr 172.21.15.77:0/2966079683 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.285+0000 7ff03da2d700 1 -- 172.21.15.77:0/2966079683 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03807d9b0 con 0x7ff03807ed80 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.286+0000 7ff03da2d700 1 --2- 172.21.15.77:0/2966079683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03807d2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0340098a0 tx=0x7ff034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08c0a6411cfe55f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.288+0000 7ff03ca2b700 1 -- 172.21.15.77:0/2966079683 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff034004160 con 0x7ff03807ed80 2024-05-27T08:28:05.073 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.288+0000 7ff03ca2b700 1 -- 172.21.15.77:0/2966079683 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff034004300 con 0x7ff03807ed80 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.288+0000 7ff03ca2b700 1 -- 172.21.15.77:0/2966079683 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff034011a20 con 0x7ff03807ed80 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.289+0000 7ff03fc91700 1 -- 172.21.15.77:0/2966079683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 msgr2=0x7ff03807d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.289+0000 7ff03fc91700 1 --2- 172.21.15.77:0/2966079683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03807d2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0340098a0 tx=0x7ff034009bb0 comp rx=0 tx=0).stop 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.289+0000 7ff03fc91700 1 -- 172.21.15.77:0/2966079683 shutdown_connections 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.289+0000 7ff03fc91700 1 --2- 172.21.15.77:0/2966079683 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03807d2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.289+0000 7ff03fc91700 1 -- 172.21.15.77:0/2966079683 >> 172.21.15.77:0/2966079683 conn(0x7ff038104720 msgr2=0x7ff038106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.289+0000 7ff03fc91700 1 -- 172.21.15.77:0/2966079683 shutdown_connections 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.289+0000 7ff03fc91700 1 -- 172.21.15.77:0/2966079683 wait complete. 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03fc91700 1 Processor -- start 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03fc91700 1 -- start start 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03fc91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:05.074 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03fc91700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff03810f530 con 0x7ff03807ed80 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03da2d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03da2d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46088/0 (socket says 172.21.15.77:46088) 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03da2d700 1 -- 172.21.15.77:0/419354577 learned_addr learned my addr 172.21.15.77:0/419354577 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03da2d700 1 -- 172.21.15.77:0/419354577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03819f640 con 0x7ff03807ed80 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03da2d700 1 --2- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03819f100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff034000c00 tx=0x7ff034011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff02effd700 1 -- 172.21.15.77:0/419354577 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff034003aa0 con 0x7ff03807ed80 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff02effd700 1 -- 172.21.15.77:0/419354577 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03401b8f0 con 0x7ff03807ed80 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03819f8d0 con 0x7ff03807ed80 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.290+0000 7ff02effd700 1 -- 172.21.15.77:0/419354577 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff034023870 con 0x7ff03807ed80 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.291+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0381a25c0 con 0x7ff03807ed80 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.291+0000 7ff02effd700 1 -- 172.21.15.77:0/419354577 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff034023b20 con 0x7ff03807ed80 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.292+0000 7ff02effd700 1 --2- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff02403e5c0 0x7ff024040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:05.075 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.292+0000 7ff02effd700 1 -- 172.21.15.77:0/419354577 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff034053f90 con 0x7ff03807ed80 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.292+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03819fdb0 con 0x7ff03807ed80 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.292+0000 7ff03d22c700 1 --2- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff02403e5c0 0x7ff024040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.292+0000 7ff03d22c700 1 --2- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff02403e5c0 0x7ff024040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff028006fd0 tx=0x7ff028006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.297+0000 7ff02effd700 1 -- 172.21.15.77:0/419354577 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff03819fdb0 con 0x7ff03807ed80 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.500+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff0381a2790 con 0x7ff03807ed80 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.501+0000 7ff02effd700 1 -- 172.21.15.77:0/419354577 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff03401f030 con 0x7ff03807ed80 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff02403e5c0 msgr2=0x7ff024040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 --2- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff02403e5c0 0x7ff024040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff028006fd0 tx=0x7ff028006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 msgr2=0x7ff03819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 --2- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03819f100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff034000c00 tx=0x7ff034011aa0 comp rx=0 tx=0).stop 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 shutdown_connections 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 --2- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff02403e5c0 0x7ff024040a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:05.076 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 --2- 172.21.15.77:0/419354577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff03807ed80 0x7ff03819f100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:05.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 >> 172.21.15.77:0/419354577 conn(0x7ff038104720 msgr2=0x7ff038105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:05.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 shutdown_connections 2024-05-27T08:28:05.077 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:04.503+0000 7ff03fc91700 1 -- 172.21.15.77:0/419354577 wait complete. 2024-05-27T08:28:06.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.526+0000 7f69323f8700 1 Processor -- start 2024-05-27T08:28:06.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.526+0000 7f69323f8700 1 -- start start 2024-05-27T08:28:06.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.526+0000 7f69323f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c10bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:06.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.526+0000 7f69323f8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f692c07cdc0 con 0x7f692c109ba0 2024-05-27T08:28:06.454 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.527+0000 7f692bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c10bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.527+0000 7f692bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c10bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46094/0 (socket says 172.21.15.77:46094) 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.527+0000 7f692bfff700 1 -- 172.21.15.77:0/1260692265 learned_addr learned my addr 172.21.15.77:0/1260692265 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.527+0000 7f692bfff700 1 -- 172.21.15.77:0/1260692265 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c10c4d0 con 0x7f692c109ba0 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.527+0000 7f692bfff700 1 --2- 172.21.15.77:0/1260692265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c10bf90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f691c0098a0 tx=0x7f691c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639c6348d2047ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.527+0000 7f692affd700 1 -- 172.21.15.77:0/1260692265 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f691c004160 con 0x7f692c109ba0 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.527+0000 7f692affd700 1 -- 172.21.15.77:0/1260692265 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c004300 con 0x7f692c109ba0 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.527+0000 7f692affd700 1 -- 172.21.15.77:0/1260692265 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f691c011a20 con 0x7f692c109ba0 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 -- 172.21.15.77:0/1260692265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 msgr2=0x7f692c10bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 --2- 172.21.15.77:0/1260692265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c10bf90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f691c0098a0 tx=0x7f691c009bb0 comp rx=0 tx=0).stop 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 -- 172.21.15.77:0/1260692265 shutdown_connections 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 --2- 172.21.15.77:0/1260692265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c10bf90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:06.455 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 -- 172.21.15.77:0/1260692265 >> 172.21.15.77:0/1260692265 conn(0x7f692c103510 msgr2=0x7f692c105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 -- 172.21.15.77:0/1260692265 shutdown_connections 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 -- 172.21.15.77:0/1260692265 wait complete. 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 Processor -- start 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.528+0000 7f69323f8700 1 -- start start 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f69323f8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f69323f8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f692c10d410 con 0x7f692c109ba0 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f692bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f692bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c19cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46110/0 (socket says 172.21.15.77:46110) 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f692bfff700 1 -- 172.21.15.77:0/1085846552 learned_addr learned my addr 172.21.15.77:0/1085846552 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f692bfff700 1 -- 172.21.15.77:0/1085846552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c19d510 con 0x7f692c109ba0 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f692bfff700 1 --2- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c19cfd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f691c0116e0 tx=0x7f691c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:06.456 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f69297fa700 1 -- 172.21.15.77:0/1085846552 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f691c003b20 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f69297fa700 1 -- 172.21.15.77:0/1085846552 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691c01b8f0 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.530+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c19d7a0 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.529+0000 7f69297fa700 1 -- 172.21.15.77:0/1085846552 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f691c0238c0 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.530+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c1a0440 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.531+0000 7f69297fa700 1 -- 172.21.15.77:0/1085846552 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f691c023a60 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.531+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692c064ba0 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.531+0000 7f69297fa700 1 --2- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f691403e570 0x7f6914040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.531+0000 7f69297fa700 1 -- 172.21.15.77:0/1085846552 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f691c054240 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.531+0000 7f692b7fe700 1 --2- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f691403e570 0x7f6914040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.531+0000 7f692b7fe700 1 --2- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f691403e570 0x7f6914040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6920006fd0 tx=0x7f6920006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.536+0000 7f69297fa700 1 -- 172.21.15.77:0/1085846552 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f691c01f030 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.851+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f692c1a09f0 con 0x7f692c109ba0 2024-05-27T08:28:06.457 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.854+0000 7f69297fa700 1 -- 172.21.15.77:0/1085846552 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f691c01b440 con 0x7f692c109ba0 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.856+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f691403e570 msgr2=0x7f6914040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 --2- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f691403e570 0x7f6914040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6920006fd0 tx=0x7f6920006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 msgr2=0x7f692c19cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 --2- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c19cfd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f691c0116e0 tx=0x7f691c011aa0 comp rx=0 tx=0).stop 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 shutdown_connections 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 --2- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f691403e570 0x7f6914040a30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 --2- 172.21.15.77:0/1085846552 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f692c109ba0 0x7f692c19cfd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 >> 172.21.15.77:0/1085846552 conn(0x7f692c103510 msgr2=0x7f692c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 shutdown_connections 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr 2024-05-27T08:28:05.857+0000 7f69323f8700 1 -- 172.21.15.77:0/1085846552 wait complete. 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-27T08:28:06.458 INFO:teuthology.orchestra.run.smithi077.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout:Or, if you are only running a single cluster on this host: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: ceph telemetry on 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout:For more information see: 2024-05-27T08:28:06.459 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:06.460 INFO:teuthology.orchestra.run.smithi077.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-27T08:28:06.460 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:06.460 INFO:teuthology.orchestra.run.smithi077.stdout:Bootstrap complete. 2024-05-27T08:28:06.496 INFO:tasks.cephadm:Fetching config... 2024-05-27T08:28:06.496 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:28:06.496 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-27T08:28:06.512 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-27T08:28:06.512 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:28:06.512 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-27T08:28:06.568 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-27T08:28:06.568 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:28:06.568 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-27T08:28:06.632 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-27T08:28:06.633 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:28:06.633 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-27T08:28:06.689 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-27T08:28:06.689 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB5Zx2ShVGx1YZnzn14ArZ9mUpWbJHHr5j+DQpjGKBPJ5CzAbe94z+PkSHddPDmQnp9jkDglTfIdCTP9+Jh/up7IkfTDgMmQ3Vp7ZsQ8blAUs9ayGZ38vxqPZyR1HqFqEY+DFTW5S8QsLSMRsY0ov7zAVrpRh2utYUV+CJXVbGINa3mlePhgyycVcls74jSmD78lLbAF01c9bKfOPZi3JANV9xFjbPyoyJ5XQRQUt6Gy4/9D8HFwMxVI0GJiuz7Iz3bKTWyy3lDYfMaYDKObnGbZCz/6OHo0+BZaF9OgmE0jZJeTyJBqkdnirO3iG1VxTriOrwb9zOuQbe26KtAiJ5Ti8wqb3eaVQ9BTt5XpDwnnGdmI+56dVqnqGj5rVQm8joDSIqSmGvfNhgWlcL8ztLzrDKdy7ZEIMqW3BEQujd5MBDpVpf/BmNLUvft/LEDHfheYavNb/EptMIXk4gxOlzRnAuz+Cux19jELrNCxbYxvwzihLRnNjhADMja9i3zk= ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T08:28:06.761 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB5Zx2ShVGx1YZnzn14ArZ9mUpWbJHHr5j+DQpjGKBPJ5CzAbe94z+PkSHddPDmQnp9jkDglTfIdCTP9+Jh/up7IkfTDgMmQ3Vp7ZsQ8blAUs9ayGZ38vxqPZyR1HqFqEY+DFTW5S8QsLSMRsY0ov7zAVrpRh2utYUV+CJXVbGINa3mlePhgyycVcls74jSmD78lLbAF01c9bKfOPZi3JANV9xFjbPyoyJ5XQRQUt6Gy4/9D8HFwMxVI0GJiuz7Iz3bKTWyy3lDYfMaYDKObnGbZCz/6OHo0+BZaF9OgmE0jZJeTyJBqkdnirO3iG1VxTriOrwb9zOuQbe26KtAiJ5Ti8wqb3eaVQ9BTt5XpDwnnGdmI+56dVqnqGj5rVQm8joDSIqSmGvfNhgWlcL8ztLzrDKdy7ZEIMqW3BEQujd5MBDpVpf/BmNLUvft/LEDHfheYavNb/EptMIXk4gxOlzRnAuz+Cux19jELrNCxbYxvwzihLRnNjhADMja9i3zk= ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:06.772 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB5Zx2ShVGx1YZnzn14ArZ9mUpWbJHHr5j+DQpjGKBPJ5CzAbe94z+PkSHddPDmQnp9jkDglTfIdCTP9+Jh/up7IkfTDgMmQ3Vp7ZsQ8blAUs9ayGZ38vxqPZyR1HqFqEY+DFTW5S8QsLSMRsY0ov7zAVrpRh2utYUV+CJXVbGINa3mlePhgyycVcls74jSmD78lLbAF01c9bKfOPZi3JANV9xFjbPyoyJ5XQRQUt6Gy4/9D8HFwMxVI0GJiuz7Iz3bKTWyy3lDYfMaYDKObnGbZCz/6OHo0+BZaF9OgmE0jZJeTyJBqkdnirO3iG1VxTriOrwb9zOuQbe26KtAiJ5Ti8wqb3eaVQ9BTt5XpDwnnGdmI+56dVqnqGj5rVQm8joDSIqSmGvfNhgWlcL8ztLzrDKdy7ZEIMqW3BEQujd5MBDpVpf/BmNLUvft/LEDHfheYavNb/EptMIXk4gxOlzRnAuz+Cux19jELrNCxbYxvwzihLRnNjhADMja9i3zk= ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T08:28:06.811 INFO:teuthology.orchestra.run.smithi092.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB5Zx2ShVGx1YZnzn14ArZ9mUpWbJHHr5j+DQpjGKBPJ5CzAbe94z+PkSHddPDmQnp9jkDglTfIdCTP9+Jh/up7IkfTDgMmQ3Vp7ZsQ8blAUs9ayGZ38vxqPZyR1HqFqEY+DFTW5S8QsLSMRsY0ov7zAVrpRh2utYUV+CJXVbGINa3mlePhgyycVcls74jSmD78lLbAF01c9bKfOPZi3JANV9xFjbPyoyJ5XQRQUt6Gy4/9D8HFwMxVI0GJiuz7Iz3bKTWyy3lDYfMaYDKObnGbZCz/6OHo0+BZaF9OgmE0jZJeTyJBqkdnirO3iG1VxTriOrwb9zOuQbe26KtAiJ5Ti8wqb3eaVQ9BTt5XpDwnnGdmI+56dVqnqGj5rVQm8joDSIqSmGvfNhgWlcL8ztLzrDKdy7ZEIMqW3BEQujd5MBDpVpf/BmNLUvft/LEDHfheYavNb/EptMIXk4gxOlzRnAuz+Cux19jELrNCxbYxvwzihLRnNjhADMja9i3zk= ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:06.823 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB5Zx2ShVGx1YZnzn14ArZ9mUpWbJHHr5j+DQpjGKBPJ5CzAbe94z+PkSHddPDmQnp9jkDglTfIdCTP9+Jh/up7IkfTDgMmQ3Vp7ZsQ8blAUs9ayGZ38vxqPZyR1HqFqEY+DFTW5S8QsLSMRsY0ov7zAVrpRh2utYUV+CJXVbGINa3mlePhgyycVcls74jSmD78lLbAF01c9bKfOPZi3JANV9xFjbPyoyJ5XQRQUt6Gy4/9D8HFwMxVI0GJiuz7Iz3bKTWyy3lDYfMaYDKObnGbZCz/6OHo0+BZaF9OgmE0jZJeTyJBqkdnirO3iG1VxTriOrwb9zOuQbe26KtAiJ5Ti8wqb3eaVQ9BTt5XpDwnnGdmI+56dVqnqGj5rVQm8joDSIqSmGvfNhgWlcL8ztLzrDKdy7ZEIMqW3BEQujd5MBDpVpf/BmNLUvft/LEDHfheYavNb/EptMIXk4gxOlzRnAuz+Cux19jELrNCxbYxvwzihLRnNjhADMja9i3zk= ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-27T08:28:06.863 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvB5Zx2ShVGx1YZnzn14ArZ9mUpWbJHHr5j+DQpjGKBPJ5CzAbe94z+PkSHddPDmQnp9jkDglTfIdCTP9+Jh/up7IkfTDgMmQ3Vp7ZsQ8blAUs9ayGZ38vxqPZyR1HqFqEY+DFTW5S8QsLSMRsY0ov7zAVrpRh2utYUV+CJXVbGINa3mlePhgyycVcls74jSmD78lLbAF01c9bKfOPZi3JANV9xFjbPyoyJ5XQRQUt6Gy4/9D8HFwMxVI0GJiuz7Iz3bKTWyy3lDYfMaYDKObnGbZCz/6OHo0+BZaF9OgmE0jZJeTyJBqkdnirO3iG1VxTriOrwb9zOuQbe26KtAiJ5Ti8wqb3eaVQ9BTt5XpDwnnGdmI+56dVqnqGj5rVQm8joDSIqSmGvfNhgWlcL8ztLzrDKdy7ZEIMqW3BEQujd5MBDpVpf/BmNLUvft/LEDHfheYavNb/EptMIXk4gxOlzRnAuz+Cux19jELrNCxbYxvwzihLRnNjhADMja9i3zk= ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:06.878 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-27T08:28:07.115 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:28:07.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:06 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1085846552' entity='client.admin' 2024-05-27T08:28:09.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.898+0000 7f61924d1700 1 -- 172.21.15.77:0/14072211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 msgr2=0x7f618c101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.898+0000 7f61924d1700 1 --2- 172.21.15.77:0/14072211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 0x7f618c101690 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f617c009910 tx=0x7f617c009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.899+0000 7f61924d1700 1 -- 172.21.15.77:0/14072211 shutdown_connections 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.899+0000 7f61924d1700 1 --2- 172.21.15.77:0/14072211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 0x7f618c101690 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.899+0000 7f61924d1700 1 -- 172.21.15.77:0/14072211 >> 172.21.15.77:0/14072211 conn(0x7f618c0fcbf0 msgr2=0x7f618c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.899+0000 7f61924d1700 1 -- 172.21.15.77:0/14072211 shutdown_connections 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.899+0000 7f61924d1700 1 -- 172.21.15.77:0/14072211 wait complete. 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.899+0000 7f61924d1700 1 Processor -- start 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.899+0000 7f61924d1700 1 -- start start 2024-05-27T08:28:09.901 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.900+0000 7f61924d1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 0x7f618c103ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:09.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.900+0000 7f61924d1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f618c108ef0 con 0x7f618c101270 2024-05-27T08:28:09.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.900+0000 7f618bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 0x7f618c103ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:09.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.900+0000 7f618bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 0x7f618c103ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60178/0 (socket says 172.21.15.77:60178) 2024-05-27T08:28:09.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.900+0000 7f618bfff700 1 -- 172.21.15.77:0/2180642627 learned_addr learned my addr 172.21.15.77:0/2180642627 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:09.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.900+0000 7f618bfff700 1 -- 172.21.15.77:0/2180642627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c104010 con 0x7f618c101270 2024-05-27T08:28:09.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.900+0000 7f618bfff700 1 --2- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 0x7f618c103ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f617c005800 tx=0x7f617c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:09.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.900+0000 7f61897fa700 1 -- 172.21.15.77:0/2180642627 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f617c027070 con 0x7f618c101270 2024-05-27T08:28:09.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.901+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c102200 con 0x7f618c101270 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.901+0000 7f61897fa700 1 -- 172.21.15.77:0/2180642627 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f617c00be10 con 0x7f618c101270 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.901+0000 7f61897fa700 1 -- 172.21.15.77:0/2180642627 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f617c00f460 con 0x7f618c101270 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.901+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c102740 con 0x7f618c101270 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.902+0000 7f61897fa700 1 -- 172.21.15.77:0/2180642627 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f617c02a070 con 0x7f618c101270 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.902+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c109670 con 0x7f618c101270 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.902+0000 7f61897fa700 1 --2- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f617403e570 0x7f6174040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.902+0000 7f61897fa700 1 -- 172.21.15.77:0/2180642627 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f617c01e030 con 0x7f618c101270 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.902+0000 7f618b7fe700 1 --2- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f617403e570 0x7f6174040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:09.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.903+0000 7f618b7fe700 1 --2- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f617403e570 0x7f6174040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6180006fd0 tx=0x7f6180006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:09.908 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:09.907+0000 7f61897fa700 1 -- 172.21.15.77:0/2180642627 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f617c013070 con 0x7f618c101270 2024-05-27T08:28:10.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.117+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f618c191970 con 0x7f618c101270 2024-05-27T08:28:10.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.120+0000 7f61897fa700 1 -- 172.21.15.77:0/2180642627 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f617c031020 con 0x7f618c101270 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f617403e570 msgr2=0x7f6174040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 --2- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f617403e570 0x7f6174040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6180006fd0 tx=0x7f6180006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 msgr2=0x7f618c103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 --2- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 0x7f618c103ad0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f617c005800 tx=0x7f617c009f60 comp rx=0 tx=0).stop 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 shutdown_connections 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 --2- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f617403e570 0x7f6174040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 --2- 172.21.15.77:0/2180642627 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f618c101270 0x7f618c103ad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 >> 172.21.15.77:0/2180642627 conn(0x7f618c0fcbf0 msgr2=0x7f618c0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 shutdown_connections 2024-05-27T08:28:10.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:10.123+0000 7f61924d1700 1 -- 172.21.15.77:0/2180642627 wait complete. 2024-05-27T08:28:10.656 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-27T08:28:10.656 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-27T08:28:11.128 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:28:11.280 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:11 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2180642627' entity='client.admin' 2024-05-27T08:28:11.280 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:11.280 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:11.280 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:28:11.280 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:12.242 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:12.242 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:12.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 -- 172.21.15.77:0/2702215028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 msgr2=0x7f4688104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:12.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 --2- 172.21.15.77:0/2702215028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 0x7f4688104ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4678009910 tx=0x7f4678009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:12.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 -- 172.21.15.77:0/2702215028 shutdown_connections 2024-05-27T08:28:12.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 --2- 172.21.15.77:0/2702215028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 0x7f4688104ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:12.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 -- 172.21.15.77:0/2702215028 >> 172.21.15.77:0/2702215028 conn(0x7f46880fe590 msgr2=0x7f46881009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:12.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 -- 172.21.15.77:0/2702215028 shutdown_connections 2024-05-27T08:28:12.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 -- 172.21.15.77:0/2702215028 wait complete. 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 Processor -- start 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.478+0000 7f468cf3b700 1 -- start start 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468cf3b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 0x7f4688198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468cf3b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4688109020 con 0x7f46881046b0 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 0x7f4688198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 0x7f4688198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60186/0 (socket says 172.21.15.77:60186) 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468659c700 1 -- 172.21.15.77:0/2826405273 learned_addr learned my addr 172.21.15.77:0/2826405273 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468659c700 1 -- 172.21.15.77:0/2826405273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4688198c50 con 0x7f46881046b0 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468659c700 1 --2- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 0x7f4688198710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4678005800 tx=0x7f4678009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f46777fe700 1 -- 172.21.15.77:0/2826405273 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4678027070 con 0x7f46881046b0 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4688198ee0 con 0x7f46881046b0 2024-05-27T08:28:12.481 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f46777fe700 1 -- 172.21.15.77:0/2826405273 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f467800be10 con 0x7f46881046b0 2024-05-27T08:28:12.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f46777fe700 1 -- 172.21.15.77:0/2826405273 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467800f460 con 0x7f46881046b0 2024-05-27T08:28:12.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.479+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f468819bbd0 con 0x7f46881046b0 2024-05-27T08:28:12.482 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.480+0000 7f46777fe700 1 -- 172.21.15.77:0/2826405273 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f467802a070 con 0x7f46881046b0 2024-05-27T08:28:12.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.480+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46881097a0 con 0x7f46881046b0 2024-05-27T08:28:12.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.481+0000 7f46777fe700 1 --2- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f467003e510 0x7f46700409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:12.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.481+0000 7f46777fe700 1 -- 172.21.15.77:0/2826405273 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f467801e030 con 0x7f46881046b0 2024-05-27T08:28:12.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.481+0000 7f4685d9b700 1 --2- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f467003e510 0x7f46700409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:12.483 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.481+0000 7f4685d9b700 1 --2- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f467003e510 0x7f46700409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f467c006fd0 tx=0x7f467c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:12.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.486+0000 7f46777fe700 1 -- 172.21.15.77:0/2826405273 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4678013070 con 0x7f46881046b0 2024-05-27T08:28:12.692 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.692+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f4688063980 con 0x7f467003e510 2024-05-27T08:28:12.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.695+0000 7f46777fe700 1 -- 172.21.15.77:0/2826405273 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4688063980 con 0x7f467003e510 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.697+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f467003e510 msgr2=0x7f46700409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.697+0000 7f468cf3b700 1 --2- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f467003e510 0x7f46700409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f467c006fd0 tx=0x7f467c006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.697+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 msgr2=0x7f4688198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.697+0000 7f468cf3b700 1 --2- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 0x7f4688198710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4678005800 tx=0x7f4678009f60 comp rx=0 tx=0).stop 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.697+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 shutdown_connections 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.697+0000 7f468cf3b700 1 --2- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f467003e510 0x7f46700409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.698+0000 7f468cf3b700 1 --2- 172.21.15.77:0/2826405273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46881046b0 0x7f4688198710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.698+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 >> 172.21.15.77:0/2826405273 conn(0x7f46880fe590 msgr2=0x7f46880ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.698+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 shutdown_connections 2024-05-27T08:28:12.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:12.698+0000 7f468cf3b700 1 -- 172.21.15.77:0/2826405273 wait complete. 2024-05-27T08:28:13.458 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi092 2024-05-27T08:28:13.459 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:28:13.459 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.conf 2024-05-27T08:28:13.478 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:28:13.478 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:28:13.534 INFO:tasks.cephadm:Adding host smithi092 to orchestrator... 2024-05-27T08:28:13.534 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch host add smithi092 2024-05-27T08:28:13.774 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:14.004 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:14.005 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:14.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:14 smithi077 ceph-mon[32007]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:28:14.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:14 smithi077 ceph-mon[32007]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:28:15.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 -- 172.21.15.77:0/1975022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 msgr2=0x7fb848103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:15.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 --2- 172.21.15.77:0/1975022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 0x7fb848103590 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb838009910 tx=0x7fb838009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:15.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 -- 172.21.15.77:0/1975022 shutdown_connections 2024-05-27T08:28:15.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 --2- 172.21.15.77:0/1975022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 0x7fb848103590 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:15.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 -- 172.21.15.77:0/1975022 >> 172.21.15.77:0/1975022 conn(0x7fb8480fea80 msgr2=0x7fb848100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:15.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 -- 172.21.15.77:0/1975022 shutdown_connections 2024-05-27T08:28:15.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 -- 172.21.15.77:0/1975022 wait complete. 2024-05-27T08:28:15.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 Processor -- start 2024-05-27T08:28:15.160 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.158+0000 7fb84e4f5700 1 -- start start 2024-05-27T08:28:15.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb84e4f5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 0x7fb84819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:15.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb84e4f5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb84810d430 con 0x7fb8481031b0 2024-05-27T08:28:15.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb847fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 0x7fb84819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:15.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb847fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 0x7fb84819e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60208/0 (socket says 172.21.15.77:60208) 2024-05-27T08:28:15.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb847fff700 1 -- 172.21.15.77:0/765847792 learned_addr learned my addr 172.21.15.77:0/765847792 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:15.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb847fff700 1 -- 172.21.15.77:0/765847792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84819eb60 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb847fff700 1 --2- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 0x7fb84819e620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8380056d0 tx=0x7fb8380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb8457fa700 1 -- 172.21.15.77:0/765847792 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb838026070 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb8457fa700 1 -- 172.21.15.77:0/765847792 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb83800bba0 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84819ee50 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb8457fa700 1 -- 172.21.15.77:0/765847792 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb838003920 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.159+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481987a0 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.160+0000 7fb8457fa700 1 -- 172.21.15.77:0/765847792 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb838029070 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.160+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb848104690 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.161+0000 7fb8457fa700 1 --2- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb83403e570 0x7fb834040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.161+0000 7fb8457fa700 1 -- 172.21.15.77:0/765847792 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb83801e070 con 0x7fb8481031b0 2024-05-27T08:28:15.163 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.161+0000 7fb8477fe700 1 --2- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb83403e570 0x7fb834040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:15.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.161+0000 7fb8477fe700 1 --2- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb83403e570 0x7fb834040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb830006fd0 tx=0x7fb830006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:15.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.165+0000 7fb8457fa700 1 -- 172.21.15.77:0/765847792 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb838013070 con 0x7fb8481031b0 2024-05-27T08:28:15.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:15.373+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}) -- 0x7fb848199cd0 con 0x7fb83403e570 2024-05-27T08:28:16.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:16 smithi077 ceph-mon[32007]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi092", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:28:17.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:17 smithi077 ceph-mon[32007]: Deploying cephadm binary to smithi092 2024-05-27T08:28:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:18 smithi077 ceph-mon[32007]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:19.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.506+0000 7fb8457fa700 1 -- 172.21.15.77:0/765847792 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fb848199cd0 con 0x7fb83403e570 2024-05-27T08:28:19.507 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi092' with addr '172.21.15.92' 2024-05-27T08:28:19.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.508+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb83403e570 msgr2=0x7fb834040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.508+0000 7fb84e4f5700 1 --2- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb83403e570 0x7fb834040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb830006fd0 tx=0x7fb830006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.508+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 msgr2=0x7fb84819e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.508+0000 7fb84e4f5700 1 --2- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 0x7fb84819e620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb8380056d0 tx=0x7fb8380057b0 comp rx=0 tx=0).stop 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.509+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 shutdown_connections 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.509+0000 7fb84e4f5700 1 --2- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb83403e570 0x7fb834040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.509+0000 7fb84e4f5700 1 --2- 172.21.15.77:0/765847792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb8481031b0 0x7fb84819e620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.509+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 >> 172.21.15.77:0/765847792 conn(0x7fb8480fea80 msgr2=0x7fb848104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.509+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 shutdown_connections 2024-05-27T08:28:19.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:19.509+0000 7fb84e4f5700 1 -- 172.21.15.77:0/765847792 wait complete. 2024-05-27T08:28:20.155 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T08:28:20.393 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:28:20.811 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:20 smithi077 ceph-mon[32007]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:20.811 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:20 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:20.811 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:20 smithi077 ceph-mon[32007]: Added host smithi092 2024-05-27T08:28:20.811 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:20 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:20.811 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:20 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:21.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:21 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:22.036 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.035+0000 7f745058c700 1 -- 172.21.15.77:0/667694785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 msgr2=0x7f7448102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.035+0000 7f745058c700 1 --2- 172.21.15.77:0/667694785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 0x7f7448102490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7444009910 tx=0x7f7444009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.035+0000 7f745058c700 1 -- 172.21.15.77:0/667694785 shutdown_connections 2024-05-27T08:28:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.035+0000 7f745058c700 1 --2- 172.21.15.77:0/667694785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 0x7f7448102490 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.035+0000 7f745058c700 1 -- 172.21.15.77:0/667694785 >> 172.21.15.77:0/667694785 conn(0x7f74480fd920 msgr2=0x7f74480ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.036+0000 7f745058c700 1 -- 172.21.15.77:0/667694785 shutdown_connections 2024-05-27T08:28:22.037 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.036+0000 7f745058c700 1 -- 172.21.15.77:0/667694785 wait complete. 2024-05-27T08:28:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.036+0000 7f745058c700 1 Processor -- start 2024-05-27T08:28:22.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.036+0000 7f745058c700 1 -- start start 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f745058c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 0x7f744819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f745058c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f744810c330 con 0x7f74481020b0 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f744e328700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 0x7f744819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f744e328700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 0x7f744819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45904/0 (socket says 172.21.15.77:45904) 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f744e328700 1 -- 172.21.15.77:0/2845414228 learned_addr learned my addr 172.21.15.77:0/2845414228 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f744e328700 1 -- 172.21.15.77:0/2845414228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744819ebc0 con 0x7f74481020b0 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f744e328700 1 --2- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 0x7f744819e680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74440056d0 tx=0x7f74440057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f743f7fe700 1 -- 172.21.15.77:0/2845414228 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7444026070 con 0x7f74481020b0 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f743f7fe700 1 -- 172.21.15.77:0/2845414228 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f744400bba0 con 0x7f74481020b0 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f743f7fe700 1 -- 172.21.15.77:0/2845414228 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7444003920 con 0x7f74481020b0 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744819eeb0 con 0x7f74481020b0 2024-05-27T08:28:22.039 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.037+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7448198800 con 0x7f74481020b0 2024-05-27T08:28:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.038+0000 7f743f7fe700 1 -- 172.21.15.77:0/2845414228 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7444029070 con 0x7f74481020b0 2024-05-27T08:28:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.038+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7448103590 con 0x7f74481020b0 2024-05-27T08:28:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.038+0000 7f743f7fe700 1 --2- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f743403e570 0x7f7434040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:22.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.038+0000 7f743f7fe700 1 -- 172.21.15.77:0/2845414228 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f744401e070 con 0x7f74481020b0 2024-05-27T08:28:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.039+0000 7f744db27700 1 --2- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f743403e570 0x7f7434040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:22.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.039+0000 7f744db27700 1 --2- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f743403e570 0x7f7434040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7438006fd0 tx=0x7f7438006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:22.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.043+0000 7f743f7fe700 1 -- 172.21.15.77:0/2845414228 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7444013070 con 0x7f74481020b0 2024-05-27T08:28:22.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.255+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7448199d30 con 0x7f743403e570 2024-05-27T08:28:22.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.256+0000 7f743f7fe700 1 -- 172.21.15.77:0/2845414228 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f7448199d30 con 0x7f743403e570 2024-05-27T08:28:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:22.257 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}] 2024-05-27T08:28:22.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.258+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f743403e570 msgr2=0x7f7434040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:22.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.258+0000 7f745058c700 1 --2- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f743403e570 0x7f7434040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7438006fd0 tx=0x7f7438006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:22.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.258+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 msgr2=0x7f744819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:22.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.258+0000 7f745058c700 1 --2- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 0x7f744819e680 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f74440056d0 tx=0x7f74440057b0 comp rx=0 tx=0).stop 2024-05-27T08:28:22.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.258+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 shutdown_connections 2024-05-27T08:28:22.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.258+0000 7f745058c700 1 --2- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f743403e570 0x7f7434040a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:22.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.258+0000 7f745058c700 1 --2- 172.21.15.77:0/2845414228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481020b0 0x7f744819e680 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:22.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.258+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 >> 172.21.15.77:0/2845414228 conn(0x7f74480fd920 msgr2=0x7f744806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:22.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.259+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 shutdown_connections 2024-05-27T08:28:22.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:22.259+0000 7f745058c700 1 -- 172.21.15.77:0/2845414228 wait complete. 2024-05-27T08:28:22.739 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-05-27T08:28:22.739 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:28:22.739 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-05-27T08:28:22.758 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:28:22.758 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:28:22.816 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-05-27T08:28:22.816 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch host add smithi179 2024-05-27T08:28:22.839 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:22 smithi077 ceph-mon[32007]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:22.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:23.050 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:28:23.825 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:23 smithi077 ceph-mon[32007]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:28:24.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.498+0000 7f61254fe700 1 -- 172.21.15.77:0/786225007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 msgr2=0x7f6120102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:24.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.498+0000 7f61254fe700 1 --2- 172.21.15.77:0/786225007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 0x7f6120102490 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6108009910 tx=0x7f6108009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:24.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.498+0000 7f61254fe700 1 -- 172.21.15.77:0/786225007 shutdown_connections 2024-05-27T08:28:24.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.498+0000 7f61254fe700 1 --2- 172.21.15.77:0/786225007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 0x7f6120102490 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:24.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.498+0000 7f61254fe700 1 -- 172.21.15.77:0/786225007 >> 172.21.15.77:0/786225007 conn(0x7f61200fd960 msgr2=0x7f61200ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:24.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.498+0000 7f61254fe700 1 -- 172.21.15.77:0/786225007 shutdown_connections 2024-05-27T08:28:24.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.498+0000 7f61254fe700 1 -- 172.21.15.77:0/786225007 wait complete. 2024-05-27T08:28:24.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.499+0000 7f61254fe700 1 Processor -- start 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.499+0000 7f61254fe700 1 -- start start 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.499+0000 7f61254fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 0x7f612019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.499+0000 7f61254fe700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f612010c330 con 0x7f61201020b0 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.499+0000 7f611effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 0x7f612019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.499+0000 7f611effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 0x7f612019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45918/0 (socket says 172.21.15.77:45918) 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.500+0000 7f611effd700 1 -- 172.21.15.77:0/1457906054 learned_addr learned my addr 172.21.15.77:0/1457906054 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.500+0000 7f611effd700 1 -- 172.21.15.77:0/1457906054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612019eb90 con 0x7f61201020b0 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.500+0000 7f611effd700 1 --2- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 0x7f612019e650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61080056d0 tx=0x7f61080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.500+0000 7f6117fff700 1 -- 172.21.15.77:0/1457906054 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6108026070 con 0x7f61201020b0 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.500+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612019ee80 con 0x7f61201020b0 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.500+0000 7f6117fff700 1 -- 172.21.15.77:0/1457906054 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610800bba0 con 0x7f61201020b0 2024-05-27T08:28:24.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.500+0000 7f6117fff700 1 -- 172.21.15.77:0/1457906054 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6108003920 con 0x7f61201020b0 2024-05-27T08:28:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.500+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61201987d0 con 0x7f61201020b0 2024-05-27T08:28:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.501+0000 7f6117fff700 1 -- 172.21.15.77:0/1457906054 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6108029070 con 0x7f61201020b0 2024-05-27T08:28:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.501+0000 7f6117fff700 1 --2- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f610c03e510 0x7f610c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:24.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.501+0000 7f6117fff700 1 -- 172.21.15.77:0/1457906054 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f610801e070 con 0x7f61201020b0 2024-05-27T08:28:24.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.501+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6120103590 con 0x7f61201020b0 2024-05-27T08:28:24.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.501+0000 7f611e7fc700 1 --2- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f610c03e510 0x7f610c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:24.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.502+0000 7f611e7fc700 1 --2- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f610c03e510 0x7f610c0409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6110006fd0 tx=0x7f6110006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:24.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.506+0000 7f6117fff700 1 -- 172.21.15.77:0/1457906054 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6108030070 con 0x7f61201020b0 2024-05-27T08:28:24.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:24.715+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) -- 0x7f6120199d00 con 0x7f610c03e510 2024-05-27T08:28:24.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:24 smithi077 ceph-mon[32007]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:25.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:25 smithi077 ceph-mon[32007]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:28:25.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:25.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:25.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:25.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:25.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:28:25.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:25.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:26.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:26 smithi077 ceph-mon[32007]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:26.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:26 smithi077 ceph-mon[32007]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-27T08:28:26.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:26 smithi077 ceph-mon[32007]: Deploying cephadm binary to smithi179 2024-05-27T08:28:26.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:26 smithi077 ceph-mon[32007]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:26.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:26 smithi077 ceph-mon[32007]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:28:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:28 smithi077 ceph-mon[32007]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:28.832 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:28.833 INFO:teuthology.orchestra.run.smithi077.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-05-27T08:28:28.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.830+0000 7f6117fff700 1 -- 172.21.15.77:0/1457906054 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6120199d00 con 0x7f610c03e510 2024-05-27T08:28:28.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.833+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f610c03e510 msgr2=0x7f610c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:28.834 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.833+0000 7f61254fe700 1 --2- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f610c03e510 0x7f610c0409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6110006fd0 tx=0x7f6110006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:28.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.833+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 msgr2=0x7f612019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:28.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.833+0000 7f61254fe700 1 --2- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 0x7f612019e650 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61080056d0 tx=0x7f61080057b0 comp rx=0 tx=0).stop 2024-05-27T08:28:28.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.833+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 shutdown_connections 2024-05-27T08:28:28.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.833+0000 7f61254fe700 1 --2- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f610c03e510 0x7f610c0409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:28.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.833+0000 7f61254fe700 1 --2- 172.21.15.77:0/1457906054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f61201020b0 0x7f612019e650 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:28.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.833+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 >> 172.21.15.77:0/1457906054 conn(0x7f61200fd960 msgr2=0x7f612006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:28.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.834+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 shutdown_connections 2024-05-27T08:28:28.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:28.834+0000 7f61254fe700 1 -- 172.21.15.77:0/1457906054 wait complete. 2024-05-27T08:28:29.459 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-05-27T08:28:29.731 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:28:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:29 smithi077 ceph-mon[32007]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:28:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:29 smithi077 ceph-mon[32007]: Added host smithi179 2024-05-27T08:28:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:31.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:30 smithi077 ceph-mon[32007]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/3855598919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 msgr2=0x7fbc08104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:31.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 --2- 172.21.15.77:0/3855598919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 0x7fbc08104590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009910 tx=0x7fbbf8009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/3855598919 shutdown_connections 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 --2- 172.21.15.77:0/3855598919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 0x7fbc08104590 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/3855598919 >> 172.21.15.77:0/3855598919 conn(0x7fbc080fd960 msgr2=0x7fbc080ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/3855598919 shutdown_connections 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/3855598919 wait complete. 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 Processor -- start 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.335+0000 7fbc0f6ca700 1 -- start start 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0f6ca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 0x7fbc0819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0f6ca700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0810d4b0 con 0x7fbc081041b0 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0d466700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 0x7fbc0819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0d466700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 0x7fbc0819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51114/0 (socket says 172.21.15.77:51114) 2024-05-27T08:28:31.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0d466700 1 -- 172.21.15.77:0/1958902125 learned_addr learned my addr 172.21.15.77:0/1958902125 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0d466700 1 -- 172.21.15.77:0/1958902125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0819ebf0 con 0x7fbc081041b0 2024-05-27T08:28:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0d466700 1 --2- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 0x7fbc0819e6b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80056d0 tx=0x7fbbf80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbbfe7fc700 1 -- 172.21.15.77:0/1958902125 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbf8026070 con 0x7fbc081041b0 2024-05-27T08:28:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0819ee80 con 0x7fbc081041b0 2024-05-27T08:28:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbbfe7fc700 1 -- 172.21.15.77:0/1958902125 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf8004370 con 0x7fbc081041b0 2024-05-27T08:28:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbbfe7fc700 1 -- 172.21.15.77:0/1958902125 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbbf80039b0 con 0x7fbc081041b0 2024-05-27T08:28:31.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.336+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc08198830 con 0x7fbc081041b0 2024-05-27T08:28:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.337+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0806bc30 con 0x7fbc081041b0 2024-05-27T08:28:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.337+0000 7fbbfe7fc700 1 -- 172.21.15.77:0/1958902125 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbbf8029070 con 0x7fbc081041b0 2024-05-27T08:28:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.337+0000 7fbbfe7fc700 1 --2- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbbf403e510 0x7fbbf40409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.338+0000 7fbbfe7fc700 1 -- 172.21.15.77:0/1958902125 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbbf801e070 con 0x7fbc081041b0 2024-05-27T08:28:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.338+0000 7fbc0cc65700 1 --2- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbbf403e510 0x7fbbf40409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:31.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.338+0000 7fbc0cc65700 1 --2- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbbf403e510 0x7fbbf40409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc04006fd0 tx=0x7fbc04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:31.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.343+0000 7fbbfe7fc700 1 -- 172.21.15.77:0/1958902125 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbbf8013070 con 0x7fbc081041b0 2024-05-27T08:28:31.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.554+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc08064ba0 con 0x7fbbf403e510 2024-05-27T08:28:31.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.555+0000 7fbbfe7fc700 1 -- 172.21.15.77:0/1958902125 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fbc08064ba0 con 0x7fbbf403e510 2024-05-27T08:28:31.556 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:28:31.556 INFO:teuthology.orchestra.run.smithi077.stdout:[{"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.92", "hostname": "smithi092", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.557+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbbf403e510 msgr2=0x7fbbf40409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 --2- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbbf403e510 0x7fbbf40409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc04006fd0 tx=0x7fbc04006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 msgr2=0x7fbc0819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 --2- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 0x7fbc0819e6b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80056d0 tx=0x7fbbf80057b0 comp rx=0 tx=0).stop 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 shutdown_connections 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 --2- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fbbf403e510 0x7fbbf40409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 --2- 172.21.15.77:0/1958902125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc081041b0 0x7fbc0819e6b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 >> 172.21.15.77:0/1958902125 conn(0x7fbc080fd960 msgr2=0x7fbc08104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 shutdown_connections 2024-05-27T08:28:31.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:31.558+0000 7fbc0f6ca700 1 -- 172.21.15.77:0/1958902125 wait complete. 2024-05-27T08:28:32.122 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-27T08:28:32.122 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-05-27T08:28:32.354 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:28:32.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:32 smithi077 ceph-mon[32007]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:32.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:32 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:32.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:32 smithi077 ceph-mon[32007]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:28:33.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.789+0000 7f3e29706700 1 -- 172.21.15.77:0/2629119909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 msgr2=0x7f3e24103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:33.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.789+0000 7f3e29706700 1 --2- 172.21.15.77:0/2629119909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 0x7f3e24103590 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009910 tx=0x7f3e14009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:33.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.789+0000 7f3e29706700 1 -- 172.21.15.77:0/2629119909 shutdown_connections 2024-05-27T08:28:33.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.789+0000 7f3e29706700 1 --2- 172.21.15.77:0/2629119909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 0x7f3e24103590 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.789+0000 7f3e29706700 1 -- 172.21.15.77:0/2629119909 >> 172.21.15.77:0/2629119909 conn(0x7f3e240fea80 msgr2=0x7f3e24100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.789+0000 7f3e29706700 1 -- 172.21.15.77:0/2629119909 shutdown_connections 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.790+0000 7f3e29706700 1 -- 172.21.15.77:0/2629119909 wait complete. 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.790+0000 7f3e29706700 1 Processor -- start 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.790+0000 7f3e29706700 1 -- start start 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.790+0000 7f3e29706700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 0x7f3e2419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.790+0000 7f3e29706700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410d430 con 0x7f3e241031b0 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 0x7f3e2419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 0x7f3e2419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51132/0 (socket says 172.21.15.77:51132) 2024-05-27T08:28:33.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e22ffd700 1 -- 172.21.15.77:0/849736752 learned_addr learned my addr 172.21.15.77:0/849736752 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:33.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e22ffd700 1 -- 172.21.15.77:0/849736752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2419eb60 con 0x7f3e241031b0 2024-05-27T08:28:33.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e22ffd700 1 --2- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 0x7f3e2419e620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e140056d0 tx=0x7f3e140057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:33.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e0bfff700 1 -- 172.21.15.77:0/849736752 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14026070 con 0x7f3e241031b0 2024-05-27T08:28:33.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e0bfff700 1 -- 172.21.15.77:0/849736752 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e1400bba0 con 0x7f3e241031b0 2024-05-27T08:28:33.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2419ee50 con 0x7f3e241031b0 2024-05-27T08:28:33.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e0bfff700 1 -- 172.21.15.77:0/849736752 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14003920 con 0x7f3e241031b0 2024-05-27T08:28:33.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.791+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e241987a0 con 0x7f3e241031b0 2024-05-27T08:28:33.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.792+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e24104690 con 0x7f3e241031b0 2024-05-27T08:28:33.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.792+0000 7f3e0bfff700 1 -- 172.21.15.77:0/849736752 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3e14029070 con 0x7f3e241031b0 2024-05-27T08:28:33.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.792+0000 7f3e0bfff700 1 --2- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e0c03e510 0x7f3e0c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:33.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.792+0000 7f3e0bfff700 1 -- 172.21.15.77:0/849736752 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e1401e070 con 0x7f3e241031b0 2024-05-27T08:28:33.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.792+0000 7f3e227fc700 1 --2- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e0c03e510 0x7f3e0c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:33.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.793+0000 7f3e227fc700 1 --2- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e0c03e510 0x7f3e0c0409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e18006fd0 tx=0x7f3e18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:33.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:33.797+0000 7f3e0bfff700 1 -- 172.21.15.77:0/849736752 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e14013070 con 0x7f3e241031b0 2024-05-27T08:28:34.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.003+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3e24199cd0 con 0x7f3e241031b0 2024-05-27T08:28:34.366 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.364+0000 7f3e0bfff700 1 -- 172.21.15.77:0/849736752 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3e1400d340 con 0x7f3e241031b0 2024-05-27T08:28:34.368 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.366+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e0c03e510 msgr2=0x7f3e0c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.366+0000 7f3e29706700 1 --2- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e0c03e510 0x7f3e0c0409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3e18006fd0 tx=0x7f3e18006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.366+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 msgr2=0x7f3e2419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.366+0000 7f3e29706700 1 --2- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 0x7f3e2419e620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3e140056d0 tx=0x7f3e140057b0 comp rx=0 tx=0).stop 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.366+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 shutdown_connections 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.366+0000 7f3e29706700 1 --2- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e0c03e510 0x7f3e0c0409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.366+0000 7f3e29706700 1 --2- 172.21.15.77:0/849736752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e241031b0 0x7f3e2419e620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.366+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 >> 172.21.15.77:0/849736752 conn(0x7f3e240fea80 msgr2=0x7f3e24104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.367+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 shutdown_connections 2024-05-27T08:28:34.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:34.367+0000 7f3e29706700 1 -- 172.21.15.77:0/849736752 wait complete. 2024-05-27T08:28:34.370 INFO:teuthology.orchestra.run.smithi077.stderr:adjusted tunables profile to default 2024-05-27T08:28:34.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:34 smithi077 ceph-mon[32007]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:34.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:34 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/849736752' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T08:28:35.036 INFO:tasks.cephadm:Adding mon.a on smithi077 2024-05-27T08:28:35.036 INFO:tasks.cephadm:Adding mon.b on smithi092 2024-05-27T08:28:35.037 INFO:tasks.cephadm:Adding mon.c on smithi179 2024-05-27T08:28:35.037 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi077:172.21.15.77=a;smithi092:172.21.15.92=b;smithi179:172.21.15.179=c' 2024-05-27T08:28:35.272 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:35.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/849736752' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T08:28:35.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T08:28:35.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-27T08:28:35.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:35 smithi077 ceph-mon[32007]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:36.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.797+0000 7febaf5b6700 1 -- 172.21.15.179:0/1688389914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 msgr2=0x7feba8105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:36.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.797+0000 7febaf5b6700 1 --2- 172.21.15.179:0/1688389914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 0x7feba8105260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7feba4009910 tx=0x7feba4009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:36.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.797+0000 7febaf5b6700 1 -- 172.21.15.179:0/1688389914 shutdown_connections 2024-05-27T08:28:36.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.797+0000 7febaf5b6700 1 --2- 172.21.15.179:0/1688389914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 0x7feba8105260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:36.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.797+0000 7febaf5b6700 1 -- 172.21.15.179:0/1688389914 >> 172.21.15.179:0/1688389914 conn(0x7feba80fd0b0 msgr2=0x7feba80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:36.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.797+0000 7febaf5b6700 1 -- 172.21.15.179:0/1688389914 shutdown_connections 2024-05-27T08:28:36.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.797+0000 7febaf5b6700 1 -- 172.21.15.179:0/1688389914 wait complete. 2024-05-27T08:28:36.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.798+0000 7febaf5b6700 1 Processor -- start 2024-05-27T08:28:36.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.798+0000 7febaf5b6700 1 -- start start 2024-05-27T08:28:36.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.798+0000 7febaf5b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 0x7feba819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:36.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.798+0000 7febaf5b6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feba810d560 con 0x7feba8104e80 2024-05-27T08:28:36.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.798+0000 7febad352700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 0x7feba819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.798+0000 7febad352700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 0x7feba819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.179:43518/0 (socket says 172.21.15.179:43518) 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.798+0000 7febad352700 1 -- 172.21.15.179:0/1437906072 learned_addr learned my addr 172.21.15.179:0/1437906072 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.799+0000 7febad352700 1 -- 172.21.15.179:0/1437906072 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba819ec40 con 0x7feba8104e80 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.799+0000 7febad352700 1 --2- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 0x7feba819e700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feba40056d0 tx=0x7feba40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.799+0000 7feb9e7fc700 1 -- 172.21.15.179:0/1437906072 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feba4026070 con 0x7feba8104e80 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.799+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba819ef30 con 0x7feba8104e80 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.799+0000 7feb9e7fc700 1 -- 172.21.15.179:0/1437906072 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba400bba0 con 0x7feba8104e80 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.799+0000 7feb9e7fc700 1 -- 172.21.15.179:0/1437906072 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feba4003920 con 0x7feba8104e80 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.799+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba81988c0 con 0x7feba8104e80 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.800+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba8104770 con 0x7feba8104e80 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.800+0000 7feb9e7fc700 1 -- 172.21.15.179:0/1437906072 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7feba4029070 con 0x7feba8104e80 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.800+0000 7feb9e7fc700 1 --2- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feb9403e510 0x7feb940409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:36.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.800+0000 7feb9e7fc700 1 -- 172.21.15.179:0/1437906072 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7feba401e070 con 0x7feba8104e80 2024-05-27T08:28:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.801+0000 7febacb51700 1 --2- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feb9403e510 0x7feb940409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:36.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.801+0000 7febacb51700 1 --2- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feb9403e510 0x7feb940409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb98006fd0 tx=0x7feb98006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:36.806 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:36.806+0000 7feb9e7fc700 1 -- 172.21.15.179:0/1437906072 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feba4013070 con 0x7feba8104e80 2024-05-27T08:28:36.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:36 smithi077 ceph-mon[32007]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:28:36.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:36 smithi077 ceph-mon[32007]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:36.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:36 smithi077 ceph-mon[32007]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:28:36.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:36.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:36.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:37.022 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.021+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi092:172.21.15.92=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}) -- 0x7feba8199df0 con 0x7feb9403e510 2024-05-27T08:28:37.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.028+0000 7feb9e7fc700 1 -- 172.21.15.179:0/1437906072 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7feba8199df0 con 0x7feb9403e510 2024-05-27T08:28:37.029 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-05-27T08:28:37.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feb9403e510 msgr2=0x7feb940409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 --2- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feb9403e510 0x7feb940409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb98006fd0 tx=0x7feb98006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 msgr2=0x7feba819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 --2- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 0x7feba819e700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feba40056d0 tx=0x7feba40057b0 comp rx=0 tx=0).stop 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 shutdown_connections 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 --2- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feb9403e510 0x7feb940409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 --2- 172.21.15.179:0/1437906072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feba8104e80 0x7feba819e700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 >> 172.21.15.179:0/1437906072 conn(0x7feba80fd0b0 msgr2=0x7feba80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 shutdown_connections 2024-05-27T08:28:37.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:37.030+0000 7febaf5b6700 1 -- 172.21.15.179:0/1437906072 wait complete. 2024-05-27T08:28:37.644 DEBUG:teuthology.orchestra.run.smithi092:mon.b> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.b.service 2024-05-27T08:28:37.647 DEBUG:teuthology.orchestra.run.smithi179:mon.c> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.c.service 2024-05-27T08:28:37.649 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T08:28:37.649 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T08:28:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi092:172.21.15.92=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:28:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi092:172.21.15.92=b;smithi179:172.21.15.179=c;count:3 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: Deploying daemon mon.c on smithi179 2024-05-27T08:28:38.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:38 smithi077 ceph-mon[32007]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:39.785 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:28:40.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:40 smithi077 ceph-mon[32007]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.796+0000 7f3e7623a700 1 -- 172.21.15.179:0/3928194611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 msgr2=0x7f3e70104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.796+0000 7f3e7623a700 1 --2- 172.21.15.179:0/3928194611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 0x7f3e70104680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009910 tx=0x7f3e60009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.797+0000 7f3e7623a700 1 -- 172.21.15.179:0/3928194611 shutdown_connections 2024-05-27T08:28:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.797+0000 7f3e7623a700 1 --2- 172.21.15.179:0/3928194611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 0x7f3e70104680 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:41.799 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.797+0000 7f3e7623a700 1 -- 172.21.15.179:0/3928194611 >> 172.21.15.179:0/3928194611 conn(0x7f3e700fda30 msgr2=0x7f3e700ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.797+0000 7f3e7623a700 1 -- 172.21.15.179:0/3928194611 shutdown_connections 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.797+0000 7f3e7623a700 1 -- 172.21.15.179:0/3928194611 wait complete. 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.797+0000 7f3e7623a700 1 Processor -- start 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.797+0000 7f3e7623a700 1 -- start start 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.798+0000 7f3e7623a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 0x7f3e70101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.798+0000 7f3e7623a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7010d5a0 con 0x7f3e701042a0 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.798+0000 7f3e6f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 0x7f3e70101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.798+0000 7f3e6f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 0x7f3e70101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.179:48402/0 (socket says 172.21.15.179:48402) 2024-05-27T08:28:41.800 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.798+0000 7f3e6f7fe700 1 -- 172.21.15.179:0/3579913071 learned_addr learned my addr 172.21.15.179:0/3579913071 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:28:41.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.798+0000 7f3e6f7fe700 1 -- 172.21.15.179:0/3579913071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e70103b40 con 0x7f3e701042a0 2024-05-27T08:28:41.801 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.798+0000 7f3e6f7fe700 1 --2- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 0x7f3e70101f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e600056d0 tx=0x7f3e600057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.799+0000 7f3e6cff9700 1 -- 172.21.15.179:0/3579913071 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e60026070 con 0x7f3e701042a0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.799+0000 7f3e6cff9700 1 -- 172.21.15.179:0/3579913071 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e60004370 con 0x7f3e701042a0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.799+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e70103dd0 con 0x7f3e701042a0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.799+0000 7f3e6cff9700 1 -- 172.21.15.179:0/3579913071 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e600039b0 con 0x7f3e701042a0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.799+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e70102610 con 0x7f3e701042a0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.800+0000 7f3e6cff9700 1 -- 172.21.15.179:0/3579913071 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3e60029070 con 0x7f3e701042a0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.800+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7006bf90 con 0x7f3e701042a0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.800+0000 7f3e6cff9700 1 --2- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e5803e510 0x7f3e580409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.800+0000 7f3e6cff9700 1 -- 172.21.15.179:0/3579913071 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3e6001e070 con 0x7f3e701042a0 2024-05-27T08:28:41.802 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.800+0000 7f3e6effd700 1 --2- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e5803e510 0x7f3e580409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:41.803 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.801+0000 7f3e6effd700 1 --2- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e5803e510 0x7f3e580409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3e64006fd0 tx=0x7f3e64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:41.807 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:41.806+0000 7f3e6cff9700 1 -- 172.21.15.179:0/3579913071 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e60013070 con 0x7f3e701042a0 2024-05-27T08:28:42.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.104+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3e70103430 con 0x7f3e701042a0 2024-05-27T08:28:42.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.105+0000 7f3e6cff9700 1 -- 172.21.15.179:0/3579913071 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f3e60030090 con 0x7f3e701042a0 2024-05-27T08:28:42.106 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:28:42.106 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","modified":"2024-05-27T08:26:52.412655Z","created":"2024-05-27T08:26:52.412655Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-27T08:28:42.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e5803e510 msgr2=0x7f3e580409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:42.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 --2- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e5803e510 0x7f3e580409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3e64006fd0 tx=0x7f3e64006e40 comp rx=0 tx=0).stop 2024-05-27T08:28:42.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 msgr2=0x7f3e70101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:42.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 --2- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 0x7f3e70101f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3e600056d0 tx=0x7f3e600057b0 comp rx=0 tx=0).stop 2024-05-27T08:28:42.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 shutdown_connections 2024-05-27T08:28:42.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 --2- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3e5803e510 0x7f3e580409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:42.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 --2- 172.21.15.179:0/3579913071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e701042a0 0x7f3e70101f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:42.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 >> 172.21.15.179:0/3579913071 conn(0x7f3e700fda30 msgr2=0x7f3e70107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:42.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 shutdown_connections 2024-05-27T08:28:42.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:42.107+0000 7f3e7623a700 1 -- 172.21.15.179:0/3579913071 wait complete. 2024-05-27T08:28:42.109 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-05-27T08:28:42.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:42 smithi179 ceph-mon[36481]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-27T08:28:42.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:42 smithi077 ceph-mon[32007]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:42.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:42 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/3579913071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T08:28:43.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:42 smithi179 bash[36433]: 04b807590ee240fed490315e497a318d58b19430543bacdb1e9a22c066855f24 2024-05-27T08:28:43.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:42 smithi179 systemd[1]: Started Ceph mon.c for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:28:43.775 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T08:28:43.775 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T08:28:44.021 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:28:47.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: mon.a calling monitor election 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: mon.c calling monitor election 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T08:28:47.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: monmap epoch 2 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: last_changed 2024-05-27T08:28:42.186444+0000 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: created 2024-05-27T08:26:52.412655+0000 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: min_mon_release 19 (squid) 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: election_strategy: 1 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: fsmap 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: mgrmap e13: x(active, since 50s) 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: overall HEALTH_OK 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:47.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: mon.a calling monitor election 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: mon.c calling monitor election 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: monmap epoch 2 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: last_changed 2024-05-27T08:28:42.186444+0000 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: created 2024-05-27T08:26:52.412655+0000 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: min_mon_release 19 (squid) 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: election_strategy: 1 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: fsmap 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: mgrmap e13: x(active, since 50s) 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: overall HEALTH_OK 2024-05-27T08:28:47.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:47.632 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:48.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.307+0000 7f3f35089700 1 -- 172.21.15.179:0/3875192009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f180035c0 msgr2=0x7f3f18005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:48.309 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.307+0000 7f3f35089700 1 --2- 172.21.15.179:0/3875192009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f180035c0 0x7f3f18005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3f30101e50 tx=0x7f3f2400feb0 comp rx=0 tx=0).stop 2024-05-27T08:28:48.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.307+0000 7f3f35089700 1 -- 172.21.15.179:0/3875192009 shutdown_connections 2024-05-27T08:28:48.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.307+0000 7f3f35089700 1 --2- 172.21.15.179:0/3875192009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f180035c0 0x7f3f18005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:48.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.307+0000 7f3f35089700 1 -- 172.21.15.179:0/3875192009 >> 172.21.15.179:0/3875192009 conn(0x7f3f300fd9f0 msgr2=0x7f3f300ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:48.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.307+0000 7f3f35089700 1 -- 172.21.15.179:0/3875192009 shutdown_connections 2024-05-27T08:28:48.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.307+0000 7f3f35089700 1 -- 172.21.15.179:0/3875192009 wait complete. 2024-05-27T08:28:48.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.308+0000 7f3f35089700 1 Processor -- start 2024-05-27T08:28:48.310 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.308+0000 7f3f35089700 1 -- start start 2024-05-27T08:28:48.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f35089700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 0x7f3f3019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:48.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f35089700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f3019ec00 0x7f3f301988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:48.311 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f35089700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30109e70 con 0x7f3f3019ec00 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f35089700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30109cf0 con 0x7f3f180035c0 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 0x7f3f3019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2ed9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 0x7f3f3019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:51078/0 (socket says 172.21.15.179:51078) 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2ed9d700 1 -- 172.21.15.179:0/2564059560 learned_addr learned my addr 172.21.15.179:0/2564059560 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2ed9d700 1 -- 172.21.15.179:0/2564059560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 msgr2=0x7f3f3019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2e59c700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f3019ec00 0x7f3f301988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2ed9d700 1 -- 172.21.15.179:0/2564059560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 msgr2=0x7f3f3019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2ed9d700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 0x7f3f3019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2ed9d700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 0x7f3f3019e6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2e59c700 1 -- 172.21.15.179:0/2564059560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 msgr2=0x7f3f3019e6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2e59c700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 0x7f3f3019e6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:48.312 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.309+0000 7f3f2e59c700 1 -- 172.21.15.179:0/2564059560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3019f240 con 0x7f3f3019ec00 2024-05-27T08:28:48.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.310+0000 7f3f2e59c700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f3019ec00 0x7f3f301988d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000b4d0 tx=0x7f3f2000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:48.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.310+0000 7f3f17fff700 1 -- 172.21.15.179:0/2564059560 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3f20011630 con 0x7f3f3019ec00 2024-05-27T08:28:48.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.310+0000 7f3f17fff700 1 -- 172.21.15.179:0/2564059560 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f20011c80 con 0x7f3f3019ec00 2024-05-27T08:28:48.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.310+0000 7f3f17fff700 1 -- 172.21.15.179:0/2564059560 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3f20010da0 con 0x7f3f3019ec00 2024-05-27T08:28:48.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.310+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f30198e10 con 0x7f3f3019ec00 2024-05-27T08:28:48.313 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.310+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301993a0 con 0x7f3f3019ec00 2024-05-27T08:28:48.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.311+0000 7f3f17fff700 1 -- 172.21.15.179:0/2564059560 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3f20018020 con 0x7f3f3019ec00 2024-05-27T08:28:48.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.311+0000 7f3f17fff700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3f1c03e300 0x7f3f1c0407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:48.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.311+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f30199710 con 0x7f3f3019ec00 2024-05-27T08:28:48.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.311+0000 7f3f17fff700 1 -- 172.21.15.179:0/2564059560 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3f200530d0 con 0x7f3f3019ec00 2024-05-27T08:28:48.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.312+0000 7f3f2ed9d700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3f1c03e300 0x7f3f1c0407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:48.314 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.312+0000 7f3f2ed9d700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3f1c03e300 0x7f3f1c0407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400fa60 tx=0x7f3f24005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:48.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.317+0000 7f3f17fff700 1 -- 172.21.15.179:0/2564059560 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f2001f020 con 0x7f3f3019ec00 2024-05-27T08:28:48.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:48 smithi179 ceph-mon[36481]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:48.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:48.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:28:48.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:48.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:48 smithi179 ceph-mon[36481]: Deploying daemon mon.b on smithi092 2024-05-27T08:28:48.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:48.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.610+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3f30199e50 con 0x7f3f3019ec00 2024-05-27T08:28:48.611 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.610+0000 7f3f17fff700 1 -- 172.21.15.179:0/2564059560 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f3f20010410 con 0x7f3f3019ec00 2024-05-27T08:28:48.611 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:28:48.611 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","modified":"2024-05-27T08:28:42.186444Z","created":"2024-05-27T08:26:52.412655Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-27T08:28:48.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3f1c03e300 msgr2=0x7f3f1c0407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:48.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3f1c03e300 0x7f3f1c0407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400fa60 tx=0x7f3f24005a90 comp rx=0 tx=0).stop 2024-05-27T08:28:48.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f3019ec00 msgr2=0x7f3f301988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:48.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f3019ec00 0x7f3f301988d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3f2000b4d0 tx=0x7f3f2000b890 comp rx=0 tx=0).stop 2024-05-27T08:28:48.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 shutdown_connections 2024-05-27T08:28:48.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3f1c03e300 0x7f3f1c0407c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:48.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3f180035c0 0x7f3f3019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:48.614 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 --2- 172.21.15.179:0/2564059560 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f3019ec00 0x7f3f301988d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:48.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.612+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 >> 172.21.15.179:0/2564059560 conn(0x7f3f300fd9f0 msgr2=0x7f3f300feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:48.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.613+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 shutdown_connections 2024-05-27T08:28:48.615 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:48.613+0000 7f3f35089700 1 -- 172.21.15.179:0/2564059560 wait complete. 2024-05-27T08:28:48.615 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-05-27T08:28:48.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:48 smithi077 ceph-mon[32007]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:48.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:48.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:28:48.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:48.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:48 smithi077 ceph-mon[32007]: Deploying daemon mon.b on smithi092 2024-05-27T08:28:48.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:49 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/2564059560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T08:28:49.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:49 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/2564059560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T08:28:50.178 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-27T08:28:50.179 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-05-27T08:28:50.421 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:28:50.542 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:50 smithi179 ceph-mon[36481]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:50.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:50 smithi077 ceph-mon[32007]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:50.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(???) e0 preinit fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:50.990 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).mds e1 new map 2024-05-27T08:28:50.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-27T08:28:50.991 INFO:journalctl@ceph.mon.b.smithi092.stdout: e1 2024-05-27T08:28:50.991 INFO:journalctl@ceph.mon.b.smithi092.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T08:28:50.991 INFO:journalctl@ceph.mon.b.smithi092.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T08:28:50.991 INFO:journalctl@ceph.mon.b.smithi092.stdout: legacy client fscid: -1 2024-05-27T08:28:50.991 INFO:journalctl@ceph.mon.b.smithi092.stdout: 2024-05-27T08:28:50.991 INFO:journalctl@ceph.mon.b.smithi092.stdout: No filesystems configured 2024-05-27T08:28:50.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-27T08:28:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-27T08:28:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-27T08:28:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:28:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi092", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Deploying cephadm binary to smithi179 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:28:51.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Added host smithi179 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.489 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/849736752' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/849736752' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-27T08:28:51.490 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi077:172.21.15.77=a;smithi092:172.21.15.92=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Saving service mon spec with placement smithi077:172.21.15.77=a;smithi092:172.21.15.92=b;smithi179:172.21.15.179=c;count:3 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:51.491 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:51.492 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.492 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:28:51.492 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:51.492 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Deploying daemon mon.c on smithi179 2024-05-27T08:28:51.492 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.492 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.492 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/3579913071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.a calling monitor election 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.c calling monitor election 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:51.493 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: monmap epoch 2 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: last_changed 2024-05-27T08:28:42.186444+0000 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: created 2024-05-27T08:26:52.412655+0000 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: min_mon_release 19 (squid) 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: election_strategy: 1 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: fsmap 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mgrmap e13: x(active, since 50s) 2024-05-27T08:28:51.494 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: overall HEALTH_OK 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: Deploying daemon mon.b on smithi092 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/2564059560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:50 smithi092 ceph-mon[36049]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:51 smithi092 bash[36030]: 7d3aeeafd63cd641285060f67660da6c2ecef7ffd274a641ffc9c13521bba940 2024-05-27T08:28:51.495 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:51 smithi092 systemd[1]: Started Ceph mon.b for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:28:56.316 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:28:56.317 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: mon.a calling monitor election 2024-05-27T08:28:56.317 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.317 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:56.317 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: mon.c calling monitor election 2024-05-27T08:28:56.317 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.317 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.317 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.317 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: mon.b calling monitor election 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: monmap epoch 3 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: last_changed 2024-05-27T08:28:50.995915+0000 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: created 2024-05-27T08:26:52.412655+0000 2024-05-27T08:28:56.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: min_mon_release 19 (squid) 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: election_strategy: 1 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: fsmap 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: mgrmap e13: x(active, since 58s) 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: overall HEALTH_OK 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.319 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.320 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.314+0000 7f1b03a58700 1 -- 172.21.15.179:0/2405012750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae80035c0 msgr2=0x7f1ae8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:56.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.314+0000 7f1b03a58700 1 --2- 172.21.15.179:0/2405012750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae80035c0 0x7f1ae8005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1afc06a2a0 tx=0x7f1af800fe60 comp rx=0 tx=0).stop 2024-05-27T08:28:56.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.314+0000 7f1b03a58700 1 -- 172.21.15.179:0/2405012750 shutdown_connections 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.314+0000 7f1b03a58700 1 --2- 172.21.15.179:0/2405012750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ae80035c0 0x7f1ae8005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.315+0000 7f1b03a58700 1 -- 172.21.15.179:0/2405012750 >> 172.21.15.179:0/2405012750 conn(0x7f1afc0fcd70 msgr2=0x7f1afc0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.315+0000 7f1b03a58700 1 -- 172.21.15.179:0/2405012750 shutdown_connections 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.315+0000 7f1b03a58700 1 -- 172.21.15.179:0/2405012750 wait complete. 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b03a58700 1 Processor -- start 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b03a58700 1 -- start start 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b03a58700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 0x7f1afc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b03a58700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc19ed90 0x7f1afc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b03a58700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1afc198f50 0x7f1afc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b03a58700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10af40 con 0x7f1afc19ed90 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b017f4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 0x7f1afc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b017f4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 0x7f1afc19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54906/0 (socket says 172.21.15.179:54906) 2024-05-27T08:28:56.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b017f4700 1 -- 172.21.15.179:0/672765751 learned_addr learned my addr 172.21.15.179:0/672765751 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b03a58700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10adc0 con 0x7f1afc198f50 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10b0c0 con 0x7f1ae80035c0 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b00ff3700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc19ed90 0x7f1afc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b017f4700 1 -- 172.21.15.179:0/672765751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 msgr2=0x7f1afc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b017f4700 1 -- 172.21.15.179:0/672765751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 msgr2=0x7f1afc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.316+0000 7f1b017f4700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 0x7f1afc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b017f4700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 0x7f1afc19e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b01ff5700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1afc198f50 0x7f1afc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b00ff3700 1 -- 172.21.15.179:0/672765751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 msgr2=0x7f1afc19e850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b00ff3700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 0x7f1afc19e850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b00ff3700 1 -- 172.21.15.179:0/672765751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1afc198f50 msgr2=0x7f1afc199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:56.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b00ff3700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1afc198f50 0x7f1afc199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b00ff3700 1 -- 172.21.15.179:0/672765751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc1a5720 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b00ff3700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc19ed90 0x7f1afc198a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00b550 tx=0x7f1aec00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1af27fc700 1 -- 172.21.15.179:0/672765751 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec004230 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1af27fc700 1 -- 172.21.15.179:0/672765751 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aec0043d0 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc1a5950 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1af27fc700 1 -- 172.21.15.179:0/672765751 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec011b90 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.317+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc1a5f00 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.318+0000 7f1af27fc700 1 -- 172.21.15.179:0/672765751 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1aec01a400 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.319+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc064ba0 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.319+0000 7f1af27fc700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1ae003e9a0 0x7f1ae0040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.319+0000 7f1af27fc700 1 -- 172.21.15.179:0/672765751 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1aec052330 con 0x7f1afc19ed90 2024-05-27T08:28:56.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.319+0000 7f1b017f4700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1ae003e9a0 0x7f1ae0040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.320+0000 7f1b017f4700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1ae003e9a0 0x7f1ae0040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1af800fa10 tx=0x7f1af80038d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:56.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.324+0000 7f1af27fc700 1 -- 172.21.15.179:0/672765751 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1aec020740 con 0x7f1afc19ed90 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: mon.a calling monitor election 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: mon.c calling monitor election 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: mon.b calling monitor election 2024-05-27T08:28:56.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: monmap epoch 3 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: last_changed 2024-05-27T08:28:50.995915+0000 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: created 2024-05-27T08:26:52.412655+0000 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: min_mon_release 19 (squid) 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: election_strategy: 1 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-05-27T08:28:56.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: fsmap 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: mgrmap e13: x(active, since 58s) 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: overall HEALTH_OK 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:28:56.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: mon.a calling monitor election 2024-05-27T08:28:56.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:28:56.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: mon.c calling monitor election 2024-05-27T08:28:56.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: mon.b calling monitor election 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: monmap epoch 3 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: last_changed 2024-05-27T08:28:50.995915+0000 2024-05-27T08:28:56.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: created 2024-05-27T08:26:52.412655+0000 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: min_mon_release 19 (squid) 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: election_strategy: 1 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: 0: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.a 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: 2: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: fsmap 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: osdmap e4: 0 total, 0 up, 0 in 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: mgrmap e13: x(active, since 58s) 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: overall HEALTH_OK 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:56.623 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.622+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1afc02e720 con 0x7f1afc19ed90 2024-05-27T08:28:56.624 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.623+0000 7f1af27fc700 1 -- 172.21.15.179:0/672765751 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f1aec0151c0 con 0x7f1afc19ed90 2024-05-27T08:28:56.624 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-05-27T08:28:56.624 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","modified":"2024-05-27T08:28:50.995915Z","created":"2024-05-27T08:26:52.412655Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:3300","nonce":0},{"type":"v1","addr":"172.21.15.92:6789","nonce":0}]},"addr":"172.21.15.92:6789/0","public_addr":"172.21.15.92:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-27T08:28:56.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1ae003e9a0 msgr2=0x7f1ae0040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:56.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1ae003e9a0 0x7f1ae0040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1af800fa10 tx=0x7f1af80038d0 comp rx=0 tx=0).stop 2024-05-27T08:28:56.627 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc19ed90 msgr2=0x7f1afc198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc19ed90 0x7f1afc198a10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00b550 tx=0x7f1aec00b910 comp rx=0 tx=0).stop 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 shutdown_connections 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1afc198f50 0x7f1afc199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f1ae003e9a0 0x7f1ae0040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ae80035c0 0x7f1afc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 --2- 172.21.15.179:0/672765751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1afc19ed90 0x7f1afc198a10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 >> 172.21.15.179:0/672765751 conn(0x7f1afc0fcd70 msgr2=0x7f1afc0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 shutdown_connections 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:28:56.626+0000 7f1b03a58700 1 -- 172.21.15.179:0/672765751 wait complete. 2024-05-27T08:28:56.628 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2024-05-27T08:28:57.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:57.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:57 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/672765751' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T08:28:57.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:57.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:57.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:57 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/672765751' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T08:28:57.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:57.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:28:57.381 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:57 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/672765751' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-27T08:28:57.382 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:28:57.399 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-27T08:28:57.399 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-05-27T08:28:57.644 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:28:58.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:58 smithi077 ceph-mon[32007]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:58.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:58.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:58.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:58.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:58 smithi092 ceph-mon[36049]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:58.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:58.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:58.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:58 smithi179 ceph-mon[36481]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:28:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.322 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.322 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.322 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:59.322 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.323 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:59.324 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:28:59 smithi077 ceph-mon[32007]: Reconfiguring daemon mon.a on smithi077 2024-05-27T08:28:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.436+0000 7f9214a54700 1 -- 172.21.15.77:0/2273781773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 msgr2=0x7f9210103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.436+0000 7f9214a54700 1 --2- 172.21.15.77:0/2273781773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 0x7f9210103590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9200009910 tx=0x7f9200009c20 comp rx=0 tx=0).stop 2024-05-27T08:28:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.436+0000 7f9214a54700 1 -- 172.21.15.77:0/2273781773 shutdown_connections 2024-05-27T08:28:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.436+0000 7f9214a54700 1 --2- 172.21.15.77:0/2273781773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 0x7f9210103590 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.436+0000 7f9214a54700 1 -- 172.21.15.77:0/2273781773 >> 172.21.15.77:0/2273781773 conn(0x7f92100fea60 msgr2=0x7f9210100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.436+0000 7f9214a54700 1 -- 172.21.15.77:0/2273781773 shutdown_connections 2024-05-27T08:28:59.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.436+0000 7f9214a54700 1 -- 172.21.15.77:0/2273781773 wait complete. 2024-05-27T08:28:59.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f9214a54700 1 Processor -- start 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f9214a54700 1 -- start start 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f9214a54700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 0x7f9210075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f9214a54700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9210075d80 0x7f9210076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f9214a54700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92101a4df0 0x7f92101a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f9214a54700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f921010d430 con 0x7f92101031b0 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f9214a54700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f921010d2b0 con 0x7f92101a4df0 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f9214a54700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f921010d5b0 con 0x7f9210075d80 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 0x7f9210075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 0x7f9210075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45226/0 (socket says 172.21.15.77:45226) 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 -- 172.21.15.77:0/1883773429 learned_addr learned my addr 172.21.15.77:0/1883773429 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920dd9b700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9210075d80 0x7f9210076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 -- 172.21.15.77:0/1883773429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9210075d80 msgr2=0x7f9210076200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:59.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9210075d80 0x7f9210076200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 -- 172.21.15.77:0/1883773429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92101a4df0 msgr2=0x7f92101a71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920ed9d700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92101a4df0 0x7f92101a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92101a4df0 0x7f92101a71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 -- 172.21.15.77:0/1883773429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92101a7910 con 0x7f92101031b0 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920ed9d700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92101a4df0 0x7f92101a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.437+0000 7f920e59c700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 0x7f9210075840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9200004f80 tx=0x7f9200005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.438+0000 7f91ff7fe700 1 -- 172.21.15.77:0/1883773429 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200026070 con 0x7f92101031b0 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.438+0000 7f91ff7fe700 1 -- 172.21.15.77:0/1883773429 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f920000b6f0 con 0x7f92101031b0 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.438+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92101a7c00 con 0x7f92101031b0 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.438+0000 7f91ff7fe700 1 -- 172.21.15.77:0/1883773429 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200003970 con 0x7f92101031b0 2024-05-27T08:28:59.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.438+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92101a8140 con 0x7f92101031b0 2024-05-27T08:28:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.439+0000 7f91ff7fe700 1 -- 172.21.15.77:0/1883773429 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f9200029070 con 0x7f92101031b0 2024-05-27T08:28:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.439+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9210064ba0 con 0x7f92101031b0 2024-05-27T08:28:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.439+0000 7f91ff7fe700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f91f803e9a0 0x7f91f8040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:28:59.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.439+0000 7f91ff7fe700 1 -- 172.21.15.77:0/1883773429 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9200053490 con 0x7f92101031b0 2024-05-27T08:28:59.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.439+0000 7f920dd9b700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f91f803e9a0 0x7f91f8040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:28:59.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.439+0000 7f920dd9b700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f91f803e9a0 0x7f91f8040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9204009850 tx=0x7f9204008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:28:59.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.444+0000 7f91ff7fe700 1 -- 172.21.15.77:0/1883773429 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9200013070 con 0x7f92101031b0 2024-05-27T08:28:59.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:59.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:59.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-27T08:28:59.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T08:28:59.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:28:59.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:28:59.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:59.488 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:28:59 smithi092 ceph-mon[36049]: Reconfiguring daemon mon.a on smithi077 2024-05-27T08:28:59.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: Reconfiguring mon.a (unknown last config time)... 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:28:59.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:28:59 smithi179 ceph-mon[36481]: Reconfiguring daemon mon.a on smithi077 2024-05-27T08:28:59.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.695+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f92101a8420 con 0x7f92101031b0 2024-05-27T08:28:59.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.695+0000 7f91ff7fe700 1 -- 172.21.15.77:0/1883773429 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f9200030090 con 0x7f92101031b0 2024-05-27T08:28:59.696 INFO:teuthology.orchestra.run.smithi077.stdout:# minimal ceph.conf for c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:59.697 INFO:teuthology.orchestra.run.smithi077.stdout:[global] 2024-05-27T08:28:59.697 INFO:teuthology.orchestra.run.smithi077.stdout: fsid = c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:28:59.697 INFO:teuthology.orchestra.run.smithi077.stdout: mon_host = [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-05-27T08:28:59.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f91f803e9a0 msgr2=0x7f91f8040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:59.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f91f803e9a0 0x7f91f8040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9204009850 tx=0x7f9204008040 comp rx=0 tx=0).stop 2024-05-27T08:28:59.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 msgr2=0x7f9210075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:28:59.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 0x7f9210075840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9200004f80 tx=0x7f9200005060 comp rx=0 tx=0).stop 2024-05-27T08:28:59.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 shutdown_connections 2024-05-27T08:28:59.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92101a4df0 0x7f92101a71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:59.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92101031b0 0x7f9210075840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:59.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f91f803e9a0 0x7f91f8040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:59.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 --2- 172.21.15.77:0/1883773429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9210075d80 0x7f9210076200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:28:59.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 >> 172.21.15.77:0/1883773429 conn(0x7f92100fea60 msgr2=0x7f9210104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:28:59.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 shutdown_connections 2024-05-27T08:28:59.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:28:59.698+0000 7f9214a54700 1 -- 172.21.15.77:0/1883773429 wait complete. 2024-05-27T08:29:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:00 smithi077 ceph-mon[32007]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:00.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:00 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1883773429' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:00.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:00 smithi092 ceph-mon[36049]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:00.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:00 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1883773429' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:00.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:00 smithi179 ceph-mon[36481]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:00.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:00 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1883773429' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:01.050 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-27T08:29:01.050 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:29:01.051 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T08:29:01.078 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:29:01.078 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:29:01.143 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:29:01.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T08:29:01.169 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:29:01.169 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:29:01.232 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:29:01.232 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-05-27T08:29:01.264 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:29:01.264 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:29:01.332 INFO:tasks.cephadm:Adding mgr.x on smithi077 2024-05-27T08:29:01.332 INFO:tasks.cephadm:Adding mgr.y on smithi092 2024-05-27T08:29:01.332 INFO:tasks.cephadm:Adding mgr.z on smithi179 2024-05-27T08:29:01.332 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi077=x;smithi092=y;smithi179=z' 2024-05-27T08:29:01.606 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:29:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:01 smithi092 ceph-mon[36049]: Reconfiguring mon.b (monmap changed)... 2024-05-27T08:29:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:29:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:29:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:01 smithi092 ceph-mon[36049]: Reconfiguring daemon mon.b on smithi092 2024-05-27T08:29:01.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:01.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:01.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:01 smithi179 ceph-mon[36481]: Reconfiguring mon.b (monmap changed)... 2024-05-27T08:29:01.954 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:29:01.955 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:29:01.955 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:01.955 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:01 smithi179 ceph-mon[36481]: Reconfiguring daemon mon.b on smithi092 2024-05-27T08:29:02.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:02.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:02.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:01 smithi077 ceph-mon[32007]: Reconfiguring mon.b (monmap changed)... 2024-05-27T08:29:02.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:29:02.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:29:02.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:02.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:01 smithi077 ceph-mon[32007]: Reconfiguring daemon mon.b on smithi092 2024-05-27T08:29:02.911 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:02 smithi179 ceph-mon[36481]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:02.912 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:02.912 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:02.912 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:29:02.912 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:29:02.912 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:02.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:02 smithi092 ceph-mon[36049]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:02.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:02.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:02.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:29:02.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:29:02.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:03.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:02 smithi077 ceph-mon[32007]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:03.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:03.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:03.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-27T08:29:03.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-27T08:29:03.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:03.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.439+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1295029070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f4101fc0 msgr2=0x7fa9f41023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:03.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.439+0000 7fa9f8ac5700 1 --2- 172.21.15.179:0/1295029070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f4101fc0 0x7fa9f41023e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009910 tx=0x7fa9e4009c20 comp rx=0 tx=0).stop 2024-05-27T08:29:03.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.439+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1295029070 shutdown_connections 2024-05-27T08:29:03.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.439+0000 7fa9f8ac5700 1 --2- 172.21.15.179:0/1295029070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f4101fc0 0x7fa9f41023e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:03.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.439+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1295029070 >> 172.21.15.179:0/1295029070 conn(0x7fa9f40fd580 msgr2=0x7fa9f40ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:29:03.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.439+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1295029070 shutdown_connections 2024-05-27T08:29:03.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.439+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1295029070 wait complete. 2024-05-27T08:29:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f8ac5700 1 Processor -- start 2024-05-27T08:29:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f8ac5700 1 -- start start 2024-05-27T08:29:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f8ac5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f4101fc0 0x7fa9f406c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f8ac5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f419b540 0x7fa9f419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f8ac5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f419dec0 0x7fa9f41a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f8ac5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4107f00 con 0x7fa9f419b540 2024-05-27T08:29:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f8ac5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4107d80 con 0x7fa9f4101fc0 2024-05-27T08:29:03.444 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f8ac5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4108080 con 0x7fa9f419dec0 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f419dec0 0x7fa9f41a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f419dec0 0x7fa9f41a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:55830/0 (socket says 172.21.15.179:55830) 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.440+0000 7fa9f2d9d700 1 -- 172.21.15.179:0/1053692178 learned_addr learned my addr 172.21.15.179:0/1053692178 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f1d9b700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f419b540 0x7fa9f419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f259c700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f4101fc0 0x7fa9f406c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f2d9d700 1 -- 172.21.15.179:0/1053692178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f4101fc0 msgr2=0x7fa9f406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f2d9d700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f4101fc0 0x7fa9f406c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f2d9d700 1 -- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f419b540 msgr2=0x7fa9f419d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f2d9d700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f419b540 0x7fa9f419d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f2d9d700 1 -- 172.21.15.179:0/1053692178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f41a0c30 con 0x7fa9f419dec0 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f259c700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f4101fc0 0x7fa9f406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:29:03.445 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f1d9b700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f419b540 0x7fa9f419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f2d9d700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f419dec0 0x7fa9f41a0360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00ba50 tx=0x7fa9ec00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9e37fe700 1 -- 172.21.15.179:0/1053692178 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec019070 con 0x7fa9f419dec0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9e37fe700 1 -- 172.21.15.179:0/1053692178 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ec014440 con 0x7fa9f419dec0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f41a0f20 con 0x7fa9f419dec0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9e37fe700 1 -- 172.21.15.179:0/1053692178 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec007590 con 0x7fa9f419dec0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.441+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41a1400 con 0x7fa9f419dec0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.442+0000 7fa9e37fe700 1 -- 172.21.15.179:0/1053692178 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa9ec01c070 con 0x7fa9f419dec0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.442+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f4064ba0 con 0x7fa9f419dec0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.442+0000 7fa9e37fe700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa9dc03e9a0 0x7fa9dc040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.442+0000 7fa9e37fe700 1 -- 172.21.15.179:0/1053692178 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa9ec051e80 con 0x7fa9f419dec0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.442+0000 7fa9f259c700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa9dc03e9a0 0x7fa9dc040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:03.446 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.443+0000 7fa9f259c700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa9dc03e9a0 0x7fa9dc040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40052c0 tx=0x7fa9e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:29:03.448 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.448+0000 7fa9e37fe700 1 -- 172.21.15.179:0/1053692178 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9ec022170 con 0x7fa9f419dec0 2024-05-27T08:29:03.681 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.680+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi092=y;smithi179=z", "target": ["mon-mgr", ""]}) -- 0x7fa9f4063a10 con 0x7fa9dc03e9a0 2024-05-27T08:29:03.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.685+0000 7fa9e37fe700 1 -- 172.21.15.179:0/1053692178 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa9f4063a10 con 0x7fa9dc03e9a0 2024-05-27T08:29:03.687 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2024-05-27T08:29:03.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa9dc03e9a0 msgr2=0x7fa9dc040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:03.689 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa9dc03e9a0 0x7fa9dc040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40052c0 tx=0x7fa9e4005fd0 comp rx=0 tx=0).stop 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f419dec0 msgr2=0x7fa9f41a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f419dec0 0x7fa9f41a0360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00ba50 tx=0x7fa9ec00be10 comp rx=0 tx=0).stop 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 shutdown_connections 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa9dc03e9a0 0x7fa9dc040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9f419dec0 0x7fa9f41a0360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa9f4101fc0 0x7fa9f406c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 --2- 172.21.15.179:0/1053692178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa9f419b540 0x7fa9f419d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 >> 172.21.15.179:0/1053692178 conn(0x7fa9f40fd580 msgr2=0x7fa9f40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 shutdown_connections 2024-05-27T08:29:03.690 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:29:03.688+0000 7fa9f8ac5700 1 -- 172.21.15.179:0/1053692178 wait complete. 2024-05-27T08:29:03.804 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:03 smithi179 ceph-mon[36481]: Reconfiguring mon.c (monmap changed)... 2024-05-27T08:29:03.804 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:03 smithi179 ceph-mon[36481]: Reconfiguring daemon mon.c on smithi179 2024-05-27T08:29:03.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:03 smithi092 ceph-mon[36049]: Reconfiguring mon.c (monmap changed)... 2024-05-27T08:29:03.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:03 smithi092 ceph-mon[36049]: Reconfiguring daemon mon.c on smithi179 2024-05-27T08:29:04.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:03 smithi077 ceph-mon[32007]: Reconfiguring mon.c (monmap changed)... 2024-05-27T08:29:04.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:03 smithi077 ceph-mon[32007]: Reconfiguring daemon mon.c on smithi179 2024-05-27T08:29:04.270 DEBUG:teuthology.orchestra.run.smithi092:mgr.y> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.y.service 2024-05-27T08:29:04.272 DEBUG:teuthology.orchestra.run.smithi179:mgr.z> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.z.service 2024-05-27T08:29:04.275 INFO:tasks.cephadm:Deploying OSDs... 2024-05-27T08:29:04.275 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:29:04.275 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T08:29:04.295 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T08:29:04.295 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-05-27T08:29:04.353 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T08:29:04.353 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:04.353 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-27T08:29:04.353 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:04.354 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:04.354 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-27 08:28:09.768440827 +0000 2024-05-27T08:29:04.354 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-27 08:22:49.462834234 +0000 2024-05-27T08:29:04.354 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-27 08:22:49.462834234 +0000 2024-05-27T08:29:04.354 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-27 08:22:49.462834234 +0000 2024-05-27T08:29:04.354 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T08:29:04.421 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-05-27T08:29:04.422 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-05-27T08:29:04.422 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.00021001 s, 2.4 MB/s 2024-05-27T08:29:04.423 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T08:29:04.482 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-05-27T08:29:04.538 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T08:29:04.539 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:04.539 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-27T08:29:04.539 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:04.539 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:04.539 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-27 08:28:09.799440200 +0000 2024-05-27T08:29:04.539 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-27 08:22:49.785827496 +0000 2024-05-27T08:29:04.539 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-27 08:22:49.785827496 +0000 2024-05-27T08:29:04.539 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-27 08:22:49.785827496 +0000 2024-05-27T08:29:04.539 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T08:29:04.604 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-05-27T08:29:04.605 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-05-27T08:29:04.605 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000194717 s, 2.6 MB/s 2024-05-27T08:29:04.606 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T08:29:04.663 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-05-27T08:29:04.719 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T08:29:04.719 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-27T08:29:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-27 08:28:09.828439613 +0000 2024-05-27T08:29:04.719 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-27 08:22:50.124820426 +0000 2024-05-27T08:29:04.720 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-27 08:22:50.124820426 +0000 2024-05-27T08:29:04.720 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-27 08:22:50.124820426 +0000 2024-05-27T08:29:04.720 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T08:29:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-05-27T08:29:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-05-27T08:29:04.783 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000179326 s, 2.9 MB/s 2024-05-27T08:29:04.784 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T08:29:04.841 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-05-27T08:29:04.896 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T08:29:04.896 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:04.896 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-27T08:29:04.896 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:04.896 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:04.896 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-27 08:28:09.857439027 +0000 2024-05-27T08:29:04.896 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-27 08:22:50.448813668 +0000 2024-05-27T08:29:04.896 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-27 08:22:50.448813668 +0000 2024-05-27T08:29:04.897 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-27 08:22:50.448813668 +0000 2024-05-27T08:29:04.897 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T08:29:04.958 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:04 smithi077 ceph-mon[32007]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:04.958 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:04 smithi077 ceph-mon[32007]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi092=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:04.958 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:04 smithi077 ceph-mon[32007]: Saving service mgr spec with placement smithi077=x;smithi092=y;smithi179=z;count:3 2024-05-27T08:29:04.958 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-05-27T08:29:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-05-27T08:29:04.960 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000175396 s, 2.9 MB/s 2024-05-27T08:29:04.962 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T08:29:04.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:04 smithi092 ceph-mon[36049]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:04.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:04 smithi092 ceph-mon[36049]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi092=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:04.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:04 smithi092 ceph-mon[36049]: Saving service mgr spec with placement smithi077=x;smithi092=y;smithi179=z;count:3 2024-05-27T08:29:04.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:05.018 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-05-27T08:29:05.018 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T08:29:05.036 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T08:29:05.037 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-05-27T08:29:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T08:29:05.094 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:05.094 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-27T08:29:05.094 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:05.094 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:05.094 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-27 08:28:50.459619345 +0000 2024-05-27T08:29:05.095 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-27 08:23:11.465006552 +0000 2024-05-27T08:29:05.095 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-27 08:23:11.465006552 +0000 2024-05-27T08:29:05.095 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-27 08:23:11.465006552 +0000 2024-05-27T08:29:05.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T08:29:05.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:04 smithi179 ceph-mon[36481]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:05.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:04 smithi179 ceph-mon[36481]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi077=x;smithi092=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:05.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:04 smithi179 ceph-mon[36481]: Saving service mgr spec with placement smithi077=x;smithi092=y;smithi179=z;count:3 2024-05-27T08:29:05.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:05.161 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-27T08:29:05.161 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-27T08:29:05.161 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000360438 s, 1.4 MB/s 2024-05-27T08:29:05.163 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T08:29:05.221 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-05-27T08:29:05.279 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T08:29:05.279 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:05.279 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-27T08:29:05.280 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:05.280 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:05.280 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-27 08:28:50.459619345 +0000 2024-05-27T08:29:05.280 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-27 08:23:11.831995862 +0000 2024-05-27T08:29:05.280 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-27 08:23:11.831995862 +0000 2024-05-27T08:29:05.280 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-27 08:23:11.831995862 +0000 2024-05-27T08:29:05.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T08:29:05.346 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-27T08:29:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-27T08:29:05.347 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000219212 s, 2.3 MB/s 2024-05-27T08:29:05.348 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T08:29:05.405 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-05-27T08:29:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T08:29:05.462 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:05.463 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-27T08:29:05.463 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:05.463 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:05.463 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-27 08:28:50.459619345 +0000 2024-05-27T08:29:05.463 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-27 08:23:12.185985551 +0000 2024-05-27T08:29:05.463 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-27 08:23:12.185985551 +0000 2024-05-27T08:29:05.463 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-27 08:23:12.185985551 +0000 2024-05-27T08:29:05.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T08:29:05.527 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-27T08:29:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-27T08:29:05.528 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.00022171 s, 2.3 MB/s 2024-05-27T08:29:05.529 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T08:29:05.585 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-05-27T08:29:05.640 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T08:29:05.640 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:05.640 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-27T08:29:05.640 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:05.640 INFO:teuthology.orchestra.run.smithi092.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:05.640 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-05-27 08:28:50.460619316 +0000 2024-05-27T08:29:05.640 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-05-27 08:23:12.520975794 +0000 2024-05-27T08:29:05.640 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-05-27 08:23:12.520975794 +0000 2024-05-27T08:29:05.641 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: 2024-05-27 08:23:12.520975794 +0000 2024-05-27T08:29:05.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T08:29:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-05-27T08:29:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-05-27T08:29:05.703 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000195767 s, 2.6 MB/s 2024-05-27T08:29:05.704 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T08:29:05.760 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-05-27T08:29:05.761 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-05-27T08:29:05.776 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-27T08:29:05.776 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-27 08:28:40.697205030 +0000 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-27 08:23:05.957479003 +0000 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-27 08:23:05.957479003 +0000 2024-05-27T08:29:05.832 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-27 08:23:05.957479003 +0000 2024-05-27T08:29:05.833 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-27T08:29:05.894 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-27T08:29:05.894 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-27T08:29:05.895 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000270618 s, 1.9 MB/s 2024-05-27T08:29:05.895 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-27T08:29:05.952 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-05-27T08:29:05.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:05.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:05.991 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:05.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:05.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:05.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:05.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:05.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T08:29:05.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:05.992 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:06.006 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-27T08:29:06.007 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:06.007 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-27T08:29:06.007 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:06.007 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:06.007 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-27 08:28:40.698205009 +0000 2024-05-27T08:29:06.007 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-27 08:23:06.288471578 +0000 2024-05-27T08:29:06.007 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-27 08:23:06.288471578 +0000 2024-05-27T08:29:06.007 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-27 08:23:06.288471578 +0000 2024-05-27T08:29:06.007 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-27T08:29:06.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:06.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:06.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:06.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:06.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:06.047 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:06.047 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:06.047 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T08:29:06.047 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:06.047 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:06.069 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-27T08:29:06.069 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-27T08:29:06.069 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000571357 s, 896 kB/s 2024-05-27T08:29:06.070 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-27T08:29:06.126 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-05-27T08:29:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:06.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T08:29:06.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:06.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-27 08:28:40.698205009 +0000 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-27 08:23:06.612464311 +0000 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-27 08:23:06.612464311 +0000 2024-05-27T08:29:06.181 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-27 08:23:06.612464311 +0000 2024-05-27T08:29:06.182 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-27T08:29:06.243 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-27T08:29:06.243 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-27T08:29:06.243 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000204361 s, 2.5 MB/s 2024-05-27T08:29:06.244 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-27T08:29:06.300 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-05-27T08:29:06.354 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-27T08:29:06.354 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-27T08:29:06.355 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-27T08:29:06.355 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:29:06.355 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-05-27T08:29:06.355 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-05-27 08:28:40.698205009 +0000 2024-05-27T08:29:06.355 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-05-27 08:23:06.941456931 +0000 2024-05-27T08:29:06.355 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-05-27 08:23:06.941456931 +0000 2024-05-27T08:29:06.355 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-05-27 08:23:06.941456931 +0000 2024-05-27T08:29:06.355 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-27T08:29:06.417 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-05-27T08:29:06.417 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-05-27T08:29:06.417 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000229489 s, 2.2 MB/s 2024-05-27T08:29:06.418 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-27T08:29:06.475 INFO:tasks.cephadm:Deploying osd.0 on smithi077 with /dev/vg_nvme/lv_4... 2024-05-27T08:29:06.475 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T08:29:06.717 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:29:06.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:06 smithi077 ceph-mon[32007]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:06.840 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:06 smithi077 ceph-mon[32007]: Deploying daemon mgr.z on smithi179 2024-05-27T08:29:06.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:06 smithi092 ceph-mon[36049]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:06.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:06 smithi092 ceph-mon[36049]: Deploying daemon mgr.z on smithi179 2024-05-27T08:29:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:06 smithi179 ceph-mon[36481]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:06 smithi179 ceph-mon[36481]: Deploying daemon mgr.z on smithi179 2024-05-27T08:29:07.479 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 systemd[1]: Starting Ceph mgr.z for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:29:07.734 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 podman[38307]: 2024-05-27 08:29:07.478403566 +0000 UTC m=+0.107109227 container create 2d5719555a423277b8ed7532b7352ab40fc644562b2a76828214a4b10ff0d0bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T08:29:07.734 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 podman[38307]: 2024-05-27 08:29:07.389149811 +0000 UTC m=+0.017855487 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:29:07.735 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 podman[38307]: 2024-05-27 08:29:07.706125371 +0000 UTC m=+0.334831051 container init 2d5719555a423277b8ed7532b7352ab40fc644562b2a76828214a4b10ff0d0bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:29:07.735 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 podman[38307]: 2024-05-27 08:29:07.709421803 +0000 UTC m=+0.338127465 container start 2d5719555a423277b8ed7532b7352ab40fc644562b2a76828214a4b10ff0d0bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, GIT_CLEAN=True) 2024-05-27T08:29:07.735 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 ceph-mgr[38326]: -- 172.21.15.179:0/3691676057 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f259ba5a0 con 0x555f2598ec00 2024-05-27T08:29:08.053 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 bash[38307]: 2d5719555a423277b8ed7532b7352ab40fc644562b2a76828214a4b10ff0d0bf 2024-05-27T08:29:08.053 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 systemd[1]: Started Ceph mgr.z for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:29:08.053 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:07 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:07.862+0000 7f88a3db4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T08:29:08.053 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:08.051+0000 7f88a3db4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:29:08.356 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:08.153+0000 7f88a3db4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T08:29:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T08:29:08.554 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T08:29:08.555 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-05-27T08:29:08.555 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-05-27T08:29:08.555 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012617 s, 831 MB/s 2024-05-27T08:29:08.555 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-05-27T08:29:08.856 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:08.461+0000 7f88a3db4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T08:29:08.856 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:08.546+0000 7f88a3db4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T08:29:09.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:09.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T08:29:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:08 smithi077 ceph-mon[32007]: Deploying daemon mgr.y on smithi092 2024-05-27T08:29:09.233 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:08 smithi092 ceph-mon[36049]: Deploying daemon mgr.y on smithi092 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:08 smithi179 ceph-mon[36481]: Deploying daemon mgr.y on smithi092 2024-05-27T08:29:09.467 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:29:09.986 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:09 smithi092 systemd[1]: Starting Ceph mgr.y for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:29:10.094 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:09.742+0000 7f88a3db4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T08:29:10.094 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:09.914+0000 7f88a3db4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T08:29:10.094 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:10.092+0000 7f88a3db4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T08:29:10.249 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:09 smithi092 podman[37407]: 2024-05-27 08:29:09.987064975 +0000 UTC m=+0.090056620 container create 25224e5b15df5041afbbeb09e18cd233dd927c3f7bda2887b0b9d8e1513cea49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T08:29:10.249 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 podman[37407]: 2024-05-27 08:29:09.91400853 +0000 UTC m=+0.017000173 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:29:10.249 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 podman[37407]: 2024-05-27 08:29:10.164310921 +0000 UTC m=+0.267302563 container init 25224e5b15df5041afbbeb09e18cd233dd927c3f7bda2887b0b9d8e1513cea49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS) 2024-05-27T08:29:10.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 podman[37407]: 2024-05-27 08:29:10.167367684 +0000 UTC m=+0.270359322 container start 25224e5b15df5041afbbeb09e18cd233dd927c3f7bda2887b0b9d8e1513cea49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-27T08:29:10.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 ceph-mgr[37427]: -- 172.21.15.92:0/2488520440 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad2eeaa5a0 con 0x55ad2ee7ec00 2024-05-27T08:29:10.250 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 bash[37407]: 25224e5b15df5041afbbeb09e18cd233dd927c3f7bda2887b0b9d8e1513cea49 2024-05-27T08:29:10.520 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 systemd[1]: Started Ceph mgr.y for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:29:10.520 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:10.321+0000 7f4101eeb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T08:29:10.713 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:10.401+0000 7f88a3db4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T08:29:10.713 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:10.497+0000 7f88a3db4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T08:29:10.713 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:10.712+0000 7f88a3db4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T08:29:10.869 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:10.524+0000 7f4101eeb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:29:10.870 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:10.635+0000 7f4101eeb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T08:29:11.013 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 -- 172.21.15.77:0/1356690468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c104060 msgr2=0x7fb64c10a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1356690468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c104060 0x7fb64c10a910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb64400b0d0 tx=0x7fb64400b3e0 comp rx=0 tx=0).stop 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 -- 172.21.15.77:0/1356690468 shutdown_connections 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1356690468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c104060 0x7fb64c10a910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1356690468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c1024c0 0x7fb64c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1356690468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c1036c0 0x7fb64c103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 -- 172.21.15.77:0/1356690468 >> 172.21.15.77:0/1356690468 conn(0x7fb64c0fda60 msgr2=0x7fb64c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 -- 172.21.15.77:0/1356690468 shutdown_connections 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 -- 172.21.15.77:0/1356690468 wait complete. 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.011+0000 7fb650f1f700 1 Processor -- start 2024-05-27T08:29:11.014 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb650f1f700 1 -- start start 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb650f1f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c1024c0 0x7fb64c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb650f1f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c1036c0 0x7fb64c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb650f1f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c104060 0x7fb64c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb650f1f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c10df40 con 0x7fb64c1024c0 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb650f1f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c10ddc0 con 0x7fb64c104060 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb650f1f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c10e0c0 con 0x7fb64c1036c0 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c1024c0 0x7fb64c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c1024c0 0x7fb64c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50356/0 (socket says 172.21.15.77:50356) 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 -- 172.21.15.77:0/1365836526 learned_addr learned my addr 172.21.15.77:0/1365836526 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb649d9b700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c1036c0 0x7fb64c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64ad9d700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c104060 0x7fb64c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 -- 172.21.15.77:0/1365836526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c1036c0 msgr2=0x7fb64c199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:11.015 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c1036c0 0x7fb64c199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 -- 172.21.15.77:0/1365836526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c104060 msgr2=0x7fb64c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c104060 0x7fb64c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 -- 172.21.15.77:0/1365836526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c1a12f0 con 0x7fb64c1024c0 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.012+0000 7fb64a59c700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c1024c0 0x7fb64c199690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00e9a0 tx=0x7fb63c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.013+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c004220 con 0x7fb64c1024c0 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.013+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63c0043c0 con 0x7fb64c1024c0 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.013+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c1a15e0 con 0x7fb64c1024c0 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.013+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c003cc0 con 0x7fb64c1024c0 2024-05-27T08:29:11.016 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.013+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c1a1b70 con 0x7fb64c1024c0 2024-05-27T08:29:11.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.013+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb63c018430 con 0x7fb64c1024c0 2024-05-27T08:29:11.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.013+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64c06a230 con 0x7fb64c1024c0 2024-05-27T08:29:11.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.014+0000 7fb63b7fe700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb63403e750 0x7fb634040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:11.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.014+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb63c052b00 con 0x7fb64c1024c0 2024-05-27T08:29:11.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.014+0000 7fb649d9b700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb63403e750 0x7fb634040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:11.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.014+0000 7fb649d9b700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb63403e750 0x7fb634040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb640005950 tx=0x7fb64000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:29:11.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.019+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb63c020c40 con 0x7fb64c1024c0 2024-05-27T08:29:11.106 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:10.917+0000 7f88a3db4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:29:11.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:10 smithi179 ceph-mon[36481]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:11.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:10 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:10 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:10 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:10 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:10 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:11.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:10 smithi077 ceph-mon[32007]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:11.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:10 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:10 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:10 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:10 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:10 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:11.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:11.229+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb64c06c380 con 0x7fb63403e750 2024-05-27T08:29:11.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:10.963+0000 7f4101eeb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T08:29:11.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:11.070+0000 7f4101eeb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T08:29:11.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:10 smithi092 ceph-mon[36049]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:11.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:10 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:10 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:10 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:10 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:11.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:10 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:11.809 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:11.576+0000 7f88a3db4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T08:29:11.809 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:11.807+0000 7f88a3db4200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T08:29:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:29:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:29:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:12.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:12.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:29:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:29:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:12.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:12.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:12.606 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:12.234+0000 7f88a3db4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T08:29:12.606 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:12.329+0000 7f88a3db4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T08:29:12.645 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:12.307+0000 7f4101eeb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T08:29:12.646 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:12.475+0000 7f4101eeb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T08:29:12.646 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:12.644+0000 7f4101eeb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T08:29:12.933 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:12 smithi092 ceph-mon[36049]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:12.933 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:12 smithi092 ceph-mon[36049]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:12.933 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:12 smithi092 ceph-mon[36049]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T08:29:12.934 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:12.934 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:12.934 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:12.934 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:12 smithi092 ceph-mon[36049]: Reconfiguring daemon mgr.x on smithi077 2024-05-27T08:29:12.934 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:13.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:12 smithi179 ceph-mon[36481]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:13.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:12 smithi179 ceph-mon[36481]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:13.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:12 smithi179 ceph-mon[36481]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T08:29:13.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:13.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:13.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:13.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:12 smithi179 ceph-mon[36481]: Reconfiguring daemon mgr.x on smithi077 2024-05-27T08:29:13.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:13.054 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:12.698+0000 7f88a3db4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T08:29:13.054 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:13.052+0000 7f88a3db4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T08:29:13.066 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:12 smithi077 ceph-mon[32007]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:13.067 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:12 smithi077 ceph-mon[32007]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:13.067 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:12 smithi077 ceph-mon[32007]: Reconfiguring mgr.x (unknown last config time)... 2024-05-27T08:29:13.067 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-27T08:29:13.067 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-27T08:29:13.067 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:13.067 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:12 smithi077 ceph-mon[32007]: Reconfiguring daemon mgr.x on smithi077 2024-05-27T08:29:13.067 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:13.227 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:12.932+0000 7f4101eeb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T08:29:13.227 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:13.024+0000 7f4101eeb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T08:29:13.227 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:13.225+0000 7f4101eeb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T08:29:13.356 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:13.146+0000 7f88a3db4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T08:29:13.485 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:13.417+0000 7f4101eeb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:29:13.856 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:13.388+0000 7f88a3db4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T08:29:13.856 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:13.520+0000 7f88a3db4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T08:29:13.856 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:13.615+0000 7f88a3db4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T08:29:14.356 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:13.943+0000 7f88a3db4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T08:29:14.356 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:14.035+0000 7f88a3db4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T08:29:14.486 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:14.016+0000 7f4101eeb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T08:29:14.486 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:14.236+0000 7f4101eeb200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T08:29:14.922 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:14.634+0000 7f88a3db4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T08:29:14.922 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:29:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:29:14.850+0000 7f88a3db4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T08:29:14.923 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:14.658+0000 7f4101eeb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T08:29:14.923 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:14.755+0000 7f4101eeb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T08:29:14.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:14.977+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fb63c01dd20 con 0x7fb64c1024c0 2024-05-27T08:29:15.235 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:15.124+0000 7f4101eeb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: Standby manager daemon z started 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T08:29:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:29:15.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T08:29:15.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:14 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:29:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:15.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: Standby manager daemon z started 2024-05-27T08:29:15.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T08:29:15.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:29:15.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T08:29:15.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:14 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: Standby manager daemon z started 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T08:29:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:14 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.179:0/99647221' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:29:15.736 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:15.476+0000 7f4101eeb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T08:29:15.736 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:15.568+0000 7f4101eeb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T08:29:16.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:15.812+0000 7f4101eeb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T08:29:16.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:15.944+0000 7f4101eeb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T08:29:16.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:16 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:16.038+0000 7f4101eeb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T08:29:16.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:16 smithi092 ceph-mon[36049]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-27T08:29:16.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T08:29:16.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:16 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2629936251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2957ea67-5b11-4557-ab6d-ec841890c830"}]: dispatch 2024-05-27T08:29:16.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:16 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2629936251' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2957ea67-5b11-4557-ab6d-ec841890c830"}]': finished 2024-05-27T08:29:16.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:16 smithi092 ceph-mon[36049]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T08:29:16.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:16.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:16 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1413014821' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:29:16.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:16 smithi179 ceph-mon[36481]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-27T08:29:16.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T08:29:16.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:16 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2629936251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2957ea67-5b11-4557-ab6d-ec841890c830"}]: dispatch 2024-05-27T08:29:16.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:16 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2629936251' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2957ea67-5b11-4557-ab6d-ec841890c830"}]': finished 2024-05-27T08:29:16.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:16 smithi179 ceph-mon[36481]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T08:29:16.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:16.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:16 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1413014821' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:29:16.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:16 smithi077 ceph-mon[32007]: mgrmap e14: x(active, since 77s), standbys: z 2024-05-27T08:29:16.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T08:29:16.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:16 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2629936251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2957ea67-5b11-4557-ab6d-ec841890c830"}]: dispatch 2024-05-27T08:29:16.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:16 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2629936251' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2957ea67-5b11-4557-ab6d-ec841890c830"}]': finished 2024-05-27T08:29:16.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:16 smithi077 ceph-mon[32007]: osdmap e5: 1 total, 0 up, 1 in 2024-05-27T08:29:16.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:16.382 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:16 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1413014821' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:29:16.736 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:16 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:16.361+0000 7f4101eeb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T08:29:16.737 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:16 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:16.454+0000 7f4101eeb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T08:29:17.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:17 smithi077 ceph-mon[32007]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:17.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:17 smithi179 ceph-mon[36481]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:17.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:17 smithi092 ceph-mon[36049]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:17.486 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:17 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:17.048+0000 7f4101eeb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T08:29:17.486 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:29:17 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:29:17.261+0000 7f4101eeb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T08:29:18.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:18.026+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fb63c017cd0 con 0x7fb64c1024c0 2024-05-27T08:29:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:18 smithi179 ceph-mon[36481]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:18 smithi179 ceph-mon[36481]: Standby manager daemon y started 2024-05-27T08:29:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:18 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T08:29:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:18 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:29:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:18 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T08:29:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:18 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:29:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:18 smithi077 ceph-mon[32007]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:18 smithi077 ceph-mon[32007]: Standby manager daemon y started 2024-05-27T08:29:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:18 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T08:29:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:18 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:29:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:18 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T08:29:18.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:18 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:29:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:18 smithi092 ceph-mon[36049]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:18 smithi092 ceph-mon[36049]: Standby manager daemon y started 2024-05-27T08:29:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:18 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T08:29:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:18 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:29:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:18 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T08:29:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:18 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.92:0/3410869513' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:29:19.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:19 smithi179 ceph-mon[36481]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-05-27T08:29:19.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:19 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T08:29:19.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:19 smithi077 ceph-mon[32007]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-05-27T08:29:19.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:19 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T08:29:19.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:19 smithi092 ceph-mon[36049]: mgrmap e15: x(active, since 80s), standbys: y, z 2024-05-27T08:29:19.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:19 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T08:29:20.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:20 smithi077 ceph-mon[32007]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:20.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:20 smithi179 ceph-mon[36481]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:20.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:20 smithi092 ceph-mon[36049]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:22.359 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:22 smithi077 ceph-mon[32007]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:22.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:22 smithi092 ceph-mon[36049]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:22.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:22 smithi179 ceph-mon[36481]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:23.411 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:23 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T08:29:23.411 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:23 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:23.411 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:23 smithi077 ceph-mon[32007]: Deploying daemon osd.0 on smithi077 2024-05-27T08:29:23.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:23 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T08:29:23.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:23 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:23.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:23 smithi092 ceph-mon[36049]: Deploying daemon osd.0 on smithi077 2024-05-27T08:29:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:23 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-27T08:29:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:23 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:23 smithi179 ceph-mon[36481]: Deploying daemon osd.0 on smithi077 2024-05-27T08:29:24.404 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:24 smithi077 ceph-mon[32007]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:24.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:24 smithi092 ceph-mon[36049]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:24.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:24 smithi179 ceph-mon[36481]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:26.470 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:26 smithi077 ceph-mon[32007]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:26.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:26 smithi092 ceph-mon[36049]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:26 smithi179 ceph-mon[36481]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:28.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:28 smithi077 ceph-mon[32007]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:28.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:28 smithi092 ceph-mon[36049]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:28.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:28 smithi179 ceph-mon[36481]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:29.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:29 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:29.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:29 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:29.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:29 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:29.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:29 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:29.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:29 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:29.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:29 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:29.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:29.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:29.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:30.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:30 smithi077 ceph-mon[32007]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:30.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:30 smithi092 ceph-mon[36049]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:30.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:30 smithi179 ceph-mon[36481]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:31.987 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:31.987 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:31.987 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:31.988 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:31.988 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.036+0000 7fb63b7fe700 1 -- 172.21.15.77:0/1365836526 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb64c06c380 con 0x7fb63403e750 2024-05-27T08:29:32.038 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 0 on host 'smithi077' 2024-05-27T08:29:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb63403e750 msgr2=0x7fb634040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb63403e750 0x7fb634040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb640005950 tx=0x7fb64000b410 comp rx=0 tx=0).stop 2024-05-27T08:29:32.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c1024c0 msgr2=0x7fb64c199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c1024c0 0x7fb64c199690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00e9a0 tx=0x7fb63c00ecb0 comp rx=0 tx=0).stop 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 shutdown_connections 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb64c104060 0x7fb64c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb64c1024c0 0x7fb64c199690 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb63403e750 0x7fb634040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 --2- 172.21.15.77:0/1365836526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb64c1036c0 0x7fb64c199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.039+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 >> 172.21.15.77:0/1365836526 conn(0x7fb64c0fda60 msgr2=0x7fb64c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.040+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 shutdown_connections 2024-05-27T08:29:32.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:29:32.040+0000 7fb650f1f700 1 -- 172.21.15.77:0/1365836526 wait complete. 2024-05-27T08:29:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.659 DEBUG:teuthology.orchestra.run.smithi077:osd.0> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.0.service 2024-05-27T08:29:32.662 INFO:tasks.cephadm:Deploying osd.1 on smithi092 with /dev/vg_nvme/lv_4... 2024-05-27T08:29:32.662 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T08:29:32.900 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:29:32.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:32 smithi077 ceph-mon[32007]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:32.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:32 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:32 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:32 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:32.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:32 smithi077 ceph-mon[32007]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T08:29:32.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:32 smithi092 ceph-mon[36049]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:32.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:32 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:32 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:32.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:32 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:32.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:32 smithi092 ceph-mon[36049]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T08:29:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:32 smithi179 ceph-mon[36481]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:32 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:32 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:32 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:32 smithi179 ceph-mon[36481]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-27T08:29:34.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:34 smithi077 ceph-mon[32007]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T08:29:34.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:34 smithi077 ceph-mon[32007]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T08:29:34.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:34 smithi077 ceph-mon[32007]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:29:34.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:34.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:34 smithi077 ceph-mon[32007]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:34.130 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:29:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0[46521]: 2024-05-27T08:29:34.030+0000 7f2ea6f0b700 -1 osd.0 0 waiting for initial osdmap 2024-05-27T08:29:34.130 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:29:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0[46521]: 2024-05-27T08:29:34.036+0000 7f2ea2cfe700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:29:34.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:34 smithi092 ceph-mon[36049]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T08:29:34.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:34 smithi092 ceph-mon[36049]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T08:29:34.226 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:34 smithi092 ceph-mon[36049]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:29:34.227 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:34.227 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:34 smithi092 ceph-mon[36049]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:34 smithi179 ceph-mon[36481]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-27T08:29:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:34 smithi179 ceph-mon[36481]: osdmap e6: 1 total, 0 up, 1 in 2024-05-27T08:29:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:34 smithi179 ceph-mon[36481]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:29:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:34 smithi179 ceph-mon[36481]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:34.797 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T08:29:34.797 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T08:29:34.797 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-27T08:29:34.797 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-27T08:29:34.797 INFO:teuthology.orchestra.run.smithi092.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122091 s, 859 MB/s 2024-05-27T08:29:34.798 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-27T08:29:35.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:35 smithi077 ceph-mon[32007]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:29:35.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:35 smithi077 ceph-mon[32007]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T08:29:35.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.135 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:35 smithi092 ceph-mon[36049]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:29:35.135 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:35 smithi092 ceph-mon[36049]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T08:29:35.135 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:35 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.135 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:35 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.135 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:35 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:35 smithi179 ceph-mon[36481]: from='osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:29:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:35 smithi179 ceph-mon[36481]: osdmap e7: 1 total, 0 up, 1 in 2024-05-27T08:29:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:35 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:35 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:35 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:35.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_4 2024-05-27T08:29:35.632 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] boot 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi077 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:29:36.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi077 to 17088M 2024-05-27T08:29:36.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:36.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:36.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:29:36.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:29:36.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] boot 2024-05-27T08:29:36.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T08:29:36.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:36.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi077 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi077 to 17088M 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:36.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:29:36.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:29:36.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: osd.0 [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] boot 2024-05-27T08:29:36.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: osdmap e8: 1 total, 1 up, 1 in 2024-05-27T08:29:36.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:29:36.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi077 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi077 to 17088M 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:36.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 -- 172.21.15.92:0/3552629740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81024b0 msgr2=0x7f74d81028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 --2- 172.21.15.92:0/3552629740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81024b0 0x7f74d81028d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f74c80099c0 tx=0x7f74c8009cd0 comp rx=0 tx=0).stop 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 -- 172.21.15.92:0/3552629740 shutdown_connections 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 --2- 172.21.15.92:0/3552629740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d8104050 0x7f74d810a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 --2- 172.21.15.92:0/3552629740 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81024b0 0x7f74d81028d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 --2- 172.21.15.92:0/3552629740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d81036b0 0x7f74d8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 -- 172.21.15.92:0/3552629740 >> 172.21.15.92:0/3552629740 conn(0x7f74d80fda70 msgr2=0x7f74d80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 -- 172.21.15.92:0/3552629740 shutdown_connections 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.131+0000 7f74e01ae700 1 -- 172.21.15.92:0/3552629740 wait complete. 2024-05-27T08:29:37.135 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74e01ae700 1 Processor -- start 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74e01ae700 1 -- start start 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74e01ae700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d81024b0 0x7f74d806c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74e01ae700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81036b0 0x7f74d819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74e01ae700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d8104050 0x7f74d81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74e01ae700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74d810e000 con 0x7f74d8104050 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74e01ae700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f74d810de80 con 0x7f74d81036b0 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74e01ae700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74d810e180 con 0x7f74d81024b0 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74dd749700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81036b0 0x7f74d819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74dd749700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81036b0 0x7f74d819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40286/0 (socket says 172.21.15.92:40286) 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74dd749700 1 -- 172.21.15.92:0/134386389 learned_addr learned my addr 172.21.15.92:0/134386389 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.132+0000 7f74de74b700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d8104050 0x7f74d81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:37.136 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74dd749700 1 -- 172.21.15.92:0/134386389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d81024b0 msgr2=0x7f74d806c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74dd749700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d81024b0 0x7f74d806c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74dd749700 1 -- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d8104050 msgr2=0x7f74d81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74dd749700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d8104050 0x7f74d81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74dd749700 1 -- 172.21.15.92:0/134386389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d81a1460 con 0x7f74d81036b0 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74dd749700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81036b0 0x7f74d819e540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f74d400d950 tx=0x7f74d400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74ceffd700 1 -- 172.21.15.92:0/134386389 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4016070 con 0x7f74d81036b0 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d81a1750 con 0x7f74d81036b0 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74ceffd700 1 -- 172.21.15.92:0/134386389 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d4009df0 con 0x7f74d81036b0 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74ceffd700 1 -- 172.21.15.92:0/134386389 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d400f3c0 con 0x7f74d81036b0 2024-05-27T08:29:37.137 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.133+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d81a1ce0 con 0x7f74d81036b0 2024-05-27T08:29:37.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.134+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d806a230 con 0x7f74d81036b0 2024-05-27T08:29:37.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.135+0000 7f74ceffd700 1 -- 172.21.15.92:0/134386389 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f74d4019070 con 0x7f74d81036b0 2024-05-27T08:29:37.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.136+0000 7f74ceffd700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74c40b27f0 0x7f74c40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:29:37.138 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.136+0000 7f74ceffd700 1 -- 172.21.15.92:0/134386389 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f74d40e0370 con 0x7f74d81036b0 2024-05-27T08:29:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.136+0000 7f74ddf4a700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74c40b27f0 0x7f74c40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:29:37.139 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.136+0000 7f74ddf4a700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74c40b27f0 0x7f74c40b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009990 tx=0x7f74c8009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:29:37.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.139+0000 7f74ceffd700 1 -- 172.21.15.92:0/134386389 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f74d40a9b00 con 0x7f74d81036b0 2024-05-27T08:29:37.352 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:37.350+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f74d8063a40 con 0x7f74c40b27f0 2024-05-27T08:29:37.352 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:37 smithi092 ceph-mon[36049]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T08:29:37.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:37 smithi179 ceph-mon[36481]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T08:29:37.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:37 smithi077 ceph-mon[32007]: osdmap e9: 1 total, 1 up, 1 in 2024-05-27T08:29:38.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:38 smithi092 ceph-mon[36049]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:38.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:38 smithi092 ceph-mon[36049]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:38.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:29:38.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:29:38.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:38 smithi179 ceph-mon[36481]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:38 smithi179 ceph-mon[36481]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:29:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:29:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:38 smithi077 ceph-mon[32007]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:38 smithi077 ceph-mon[32007]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:29:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:29:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:29:38.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:40.430 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:40 smithi092 ceph-mon[36049]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:40.430 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:40 smithi092 ceph-mon[36049]: from='client.? 172.21.15.92:0/3649461566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]: dispatch 2024-05-27T08:29:40.430 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:40 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]: dispatch 2024-05-27T08:29:40.430 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:40 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]': finished 2024-05-27T08:29:40.430 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:40 smithi092 ceph-mon[36049]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T08:29:40.430 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:40 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:40 smithi179 ceph-mon[36481]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:40 smithi179 ceph-mon[36481]: from='client.? 172.21.15.92:0/3649461566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]: dispatch 2024-05-27T08:29:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:40 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]: dispatch 2024-05-27T08:29:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:40 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]': finished 2024-05-27T08:29:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:40 smithi179 ceph-mon[36481]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T08:29:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:40 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:40.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:40 smithi077 ceph-mon[32007]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:40.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:40 smithi077 ceph-mon[32007]: from='client.? 172.21.15.92:0/3649461566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]: dispatch 2024-05-27T08:29:40.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:40 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]: dispatch 2024-05-27T08:29:40.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:40 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "128362ac-83fc-46b5-a7aa-a7c48e49c0d2"}]': finished 2024-05-27T08:29:40.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:40 smithi077 ceph-mon[32007]: osdmap e10: 2 total, 1 up, 2 in 2024-05-27T08:29:40.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:40 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:41.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:41 smithi092 ceph-mon[36049]: from='client.? 172.21.15.92:0/2380801770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:29:41.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:41 smithi179 ceph-mon[36481]: from='client.? 172.21.15.92:0/2380801770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:29:41.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:41 smithi077 ceph-mon[32007]: from='client.? 172.21.15.92:0/2380801770' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:29:42.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:42 smithi077 ceph-mon[32007]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:42.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:42 smithi092 ceph-mon[36049]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:42.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:42 smithi179 ceph-mon[36481]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:44.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:44 smithi092 ceph-mon[36049]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:44 smithi179 ceph-mon[36481]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:44.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:44 smithi077 ceph-mon[32007]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:46.479 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:46 smithi092 ceph-mon[36049]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:46.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:46 smithi179 ceph-mon[36481]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:46.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:46 smithi077 ceph-mon[32007]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:48.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:48 smithi092 ceph-mon[36049]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:48.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T08:29:48.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:48.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:48 smithi092 ceph-mon[36049]: Deploying daemon osd.1 on smithi092 2024-05-27T08:29:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:48 smithi179 ceph-mon[36481]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T08:29:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:48 smithi179 ceph-mon[36481]: Deploying daemon osd.1 on smithi092 2024-05-27T08:29:48.432 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:48 smithi077 ceph-mon[32007]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:48.432 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-27T08:29:48.432 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:48.432 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:48 smithi077 ceph-mon[32007]: Deploying daemon osd.1 on smithi092 2024-05-27T08:29:50.430 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:50 smithi092 ceph-mon[36049]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:50 smithi179 ceph-mon[36481]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:50.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:50 smithi077 ceph-mon[32007]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:52.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:52 smithi092 ceph-mon[36049]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:52 smithi179 ceph-mon[36481]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:52.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:52 smithi077 ceph-mon[32007]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:53.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:53 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:53.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:53 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:53.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:53 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:54.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:53 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:54.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:53 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:54.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:53 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:54.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:53 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:54.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:53 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:54.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:53 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:54.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:54 smithi092 ceph-mon[36049]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:55.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:54 smithi179 ceph-mon[36481]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:55.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:54 smithi077 ceph-mon[32007]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:56.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:56 smithi092 ceph-mon[36049]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:56.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:56.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:56.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:56.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:56.298 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:56.366 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.365+0000 7f74ceffd700 1 -- 172.21.15.92:0/134386389 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f74d8063a40 con 0x7f74c40b27f0 2024-05-27T08:29:56.366 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 1 on host 'smithi092' 2024-05-27T08:29:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74c40b27f0 msgr2=0x7f74c40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74c40b27f0 0x7f74c40b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009990 tx=0x7f74c8009f10 comp rx=0 tx=0).stop 2024-05-27T08:29:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81036b0 msgr2=0x7f74d819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:29:56.369 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81036b0 0x7f74d819e540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f74d400d950 tx=0x7f74d400dc60 comp rx=0 tx=0).stop 2024-05-27T08:29:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 shutdown_connections 2024-05-27T08:29:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d8104050 0x7f74d81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74c40b27f0 0x7f74c40b4cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74d81024b0 0x7f74d806c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 --2- 172.21.15.92:0/134386389 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74d81036b0 0x7f74d819e540 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:29:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 >> 172.21.15.92:0/134386389 conn(0x7f74d80fda70 msgr2=0x7f74d8106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:29:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 shutdown_connections 2024-05-27T08:29:56.370 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:29:56.368+0000 7f74e01ae700 1 -- 172.21.15.92:0/134386389 wait complete. 2024-05-27T08:29:56.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:56 smithi179 ceph-mon[36481]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:56.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:56.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:56.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:56.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:56.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:56.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:56 smithi077 ceph-mon[32007]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:56.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:56.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:56.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:29:56.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:29:56.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:57.031 DEBUG:teuthology.orchestra.run.smithi092:osd.1> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.1.service 2024-05-27T08:29:57.033 INFO:tasks.cephadm:Deploying osd.2 on smithi179 with /dev/vg_nvme/lv_4... 2024-05-27T08:29:57.033 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-27T08:29:57.277 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:29:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:57 smithi179 ceph-mon[36481]: from='osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T08:29:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:57 smithi179 ceph-mon[36481]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T08:29:57.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:57.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:57.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:57.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:57 smithi077 ceph-mon[32007]: from='osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T08:29:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:57 smithi077 ceph-mon[32007]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T08:29:57.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:57.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:29:57.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:29:57.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:57 smithi092 ceph-mon[36049]: from='osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T08:29:57.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:57 smithi092 ceph-mon[36049]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-27T08:29:58.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:58 smithi092 ceph-mon[36049]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:58.393 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:58 smithi092 ceph-mon[36049]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T08:29:58.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:58 smithi092 ceph-mon[36049]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T08:29:58.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:58.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:58 smithi092 ceph-mon[36049]: from='osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:29:58.394 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:58 smithi092 ceph-mon[36049]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:29:58.581 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:58 smithi179 ceph-mon[36481]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:58.581 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:58 smithi179 ceph-mon[36481]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T08:29:58.581 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:58 smithi179 ceph-mon[36481]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T08:29:58.581 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:58.581 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:58 smithi179 ceph-mon[36481]: from='osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:29:58.581 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:58 smithi179 ceph-mon[36481]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:29:58.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:58 smithi077 ceph-mon[32007]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:29:58.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:58 smithi077 ceph-mon[32007]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-27T08:29:58.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:58 smithi077 ceph-mon[32007]: osdmap e11: 2 total, 1 up, 2 in 2024-05-27T08:29:58.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:58.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:58 smithi077 ceph-mon[32007]: from='osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:29:58.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:58 smithi077 ceph-mon[32007]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:29:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-27T08:29:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-27T08:29:59.129 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-05-27T08:29:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-05-27T08:29:59.129 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011479 s, 913 MB/s 2024-05-27T08:29:59.129 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-05-27T08:29:59.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:59 smithi179 ceph-mon[36481]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:29:59.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:59 smithi179 ceph-mon[36481]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T08:29:59.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:59.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:59.436 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:29:59 smithi179 ceph-mon[36481]: from='osd.1 ' entity='osd.1' 2024-05-27T08:29:59.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:59 smithi077 ceph-mon[32007]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:29:59.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:59 smithi077 ceph-mon[32007]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T08:29:59.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:59.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:29:59 smithi077 ceph-mon[32007]: from='osd.1 ' entity='osd.1' 2024-05-27T08:29:59.681 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:59 smithi092 ceph-mon[36049]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:29:59.681 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:59 smithi092 ceph-mon[36049]: osdmap e12: 2 total, 1 up, 2 in 2024-05-27T08:29:59.681 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:59.681 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:29:59.681 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:29:59 smithi092 ceph-mon[36049]: from='osd.1 ' entity='osd.1' 2024-05-27T08:29:59.682 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:29:59 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1[40626]: 2024-05-27T08:29:59.276+0000 7f4ea1c4d700 -1 osd.1 0 waiting for initial osdmap 2024-05-27T08:29:59.682 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:29:59 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1[40626]: 2024-05-27T08:29:59.414+0000 7f4e99a38700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:29:59.838 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-05-27T08:30:00.080 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:30:00.444 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: overall HEALTH_OK 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.445 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:30:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:30:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:30:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: overall HEALTH_OK 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: overall HEALTH_OK 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:00.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:00.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:00.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:01.401 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:01 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi092 2024-05-27T08:30:01.401 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:01 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-27T08:30:01.401 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:01 smithi179 ceph-mon[36481]: osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] boot 2024-05-27T08:30:01.401 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:01 smithi179 ceph-mon[36481]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T08:30:01.401 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:01.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 -- 172.21.15.179:0/3961362580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41036b0 msgr2=0x7fefd4103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:01.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3961362580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41036b0 0x7fefd4103b10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009990 tx=0x7fefc4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:30:01.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 -- 172.21.15.179:0/3961362580 shutdown_connections 2024-05-27T08:30:01.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3961362580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd4104050 0x7fefd410a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:01.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3961362580 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd41024b0 0x7fefd41028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:01.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3961362580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41036b0 0x7fefd4103b10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:01.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 -- 172.21.15.179:0/3961362580 >> 172.21.15.179:0/3961362580 conn(0x7fefd40fda70 msgr2=0x7fefd40ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:30:01.597 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 -- 172.21.15.179:0/3961362580 shutdown_connections 2024-05-27T08:30:01.598 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.595+0000 7fefd8e60700 1 -- 172.21.15.179:0/3961362580 wait complete. 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.596+0000 7fefd8e60700 1 Processor -- start 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.596+0000 7fefd8e60700 1 -- start start 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.596+0000 7fefd8e60700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41024b0 0x7fefd4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.596+0000 7fefd8e60700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd41036b0 0x7fefd4199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.596+0000 7fefd8e60700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd4104050 0x7fefd41a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.596+0000 7fefd8e60700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fefd410df50 con 0x7fefd41036b0 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.596+0000 7fefd8e60700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fefd410ddd0 con 0x7fefd4104050 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.596+0000 7fefd8e60700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fefd410e0d0 con 0x7fefd41024b0 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41024b0 0x7fefd4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41024b0 0x7fefd4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:35224/0 (socket says 172.21.15.179:35224) 2024-05-27T08:30:01.600 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 -- 172.21.15.179:0/3274878686 learned_addr learned my addr 172.21.15.179:0/3274878686 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd1d9b700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd41036b0 0x7fefd4199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd2d9d700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd4104050 0x7fefd41a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 -- 172.21.15.179:0/3274878686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd4104050 msgr2=0x7fefd41a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd4104050 0x7fefd41a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 -- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd41036b0 msgr2=0x7fefd4199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd41036b0 0x7fefd4199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 -- 172.21.15.179:0/3274878686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd41a13c0 con 0x7fefd41024b0 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd2d9d700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd4104050 0x7fefd41a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd259c700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41024b0 0x7fefd4199760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fefc800d9b0 tx=0x7fefc800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefc37fe700 1 -- 172.21.15.179:0/3274878686 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8017070 con 0x7fefd41024b0 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd41a16b0 con 0x7fefd41024b0 2024-05-27T08:30:01.601 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefc37fe700 1 -- 172.21.15.179:0/3274878686 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefc8009de0 con 0x7fefd41024b0 2024-05-27T08:30:01.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.597+0000 7fefc37fe700 1 -- 172.21.15.179:0/3274878686 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc800f3c0 con 0x7fefd41024b0 2024-05-27T08:30:01.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.598+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd41a1ba0 con 0x7fefd41024b0 2024-05-27T08:30:01.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.599+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd406a230 con 0x7fefd41024b0 2024-05-27T08:30:01.602 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.600+0000 7fefc37fe700 1 -- 172.21.15.179:0/3274878686 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fefc8009930 con 0x7fefd41024b0 2024-05-27T08:30:01.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.600+0000 7fefc37fe700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fefbc0b27f0 0x7fefbc0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:01.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.600+0000 7fefc37fe700 1 -- 172.21.15.179:0/3274878686 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fefc80dfe00 con 0x7fefd41024b0 2024-05-27T08:30:01.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.600+0000 7fefd1d9b700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fefbc0b27f0 0x7fefbc0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:01.603 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.601+0000 7fefd1d9b700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fefbc0b27f0 0x7fefbc0b4cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fefc400b5c0 tx=0x7fefc4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:30:01.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.605+0000 7fefc37fe700 1 -- 172.21.15.179:0/3274878686 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fefc80a9370 con 0x7fefd41024b0 2024-05-27T08:30:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:01 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi092 2024-05-27T08:30:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:01 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-27T08:30:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:01 smithi077 ceph-mon[32007]: osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] boot 2024-05-27T08:30:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:01 smithi077 ceph-mon[32007]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T08:30:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:01 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi092 2024-05-27T08:30:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:01 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi092 to 17088M 2024-05-27T08:30:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:01 smithi092 ceph-mon[36049]: osd.1 [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] boot 2024-05-27T08:30:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:01 smithi092 ceph-mon[36049]: osdmap e13: 2 total, 2 up, 2 in 2024-05-27T08:30:01.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:30:01.817 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:01.816+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fefd406c3d0 con 0x7fefbc0b27f0 2024-05-27T08:30:02.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:02 smithi077 ceph-mon[32007]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:30:02.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:02 smithi077 ceph-mon[32007]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T08:30:02.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:02 smithi077 ceph-mon[32007]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:02.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:02.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:02.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:02.663 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:02 smithi179 ceph-mon[36481]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:30:02.663 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:02 smithi179 ceph-mon[36481]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T08:30:02.664 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:02 smithi179 ceph-mon[36481]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:02.664 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:02.664 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:02.664 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:02.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:02 smithi092 ceph-mon[36049]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-27T08:30:02.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:02 smithi092 ceph-mon[36049]: osdmap e14: 2 total, 2 up, 2 in 2024-05-27T08:30:02.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:02 smithi092 ceph-mon[36049]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:02.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:02.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:02.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:04 smithi077 ceph-mon[32007]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:04 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/2391074733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]: dispatch 2024-05-27T08:30:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:04 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]: dispatch 2024-05-27T08:30:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:04 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]': finished 2024-05-27T08:30:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:04 smithi077 ceph-mon[32007]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T08:30:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:04.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:04 smithi179 ceph-mon[36481]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:04.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:04 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/2391074733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]: dispatch 2024-05-27T08:30:04.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:04 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]: dispatch 2024-05-27T08:30:04.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:04 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]': finished 2024-05-27T08:30:04.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:04 smithi179 ceph-mon[36481]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T08:30:04.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:04.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:04 smithi092 ceph-mon[36049]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:04.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:04 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/2391074733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]: dispatch 2024-05-27T08:30:04.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:04 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]: dispatch 2024-05-27T08:30:04.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:04 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae8a416-499c-4a05-9389-9dac4e5415de"}]': finished 2024-05-27T08:30:04.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:04 smithi092 ceph-mon[36049]: osdmap e15: 3 total, 2 up, 3 in 2024-05-27T08:30:04.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:05.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:05 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/1602416682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:05.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:05 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/1602416682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:05.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:05 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/1602416682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:06.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:06 smithi092 ceph-mon[36049]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:06.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:06 smithi179 ceph-mon[36481]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:06.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:06 smithi077 ceph-mon[32007]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:08.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:08 smithi179 ceph-mon[36481]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:08.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:08 smithi092 ceph-mon[36049]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:08.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:08 smithi077 ceph-mon[32007]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:10.506 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:10 smithi179 ceph-mon[36481]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:10.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:10 smithi092 ceph-mon[36049]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:10.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:10 smithi077 ceph-mon[32007]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:12.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:12 smithi179 ceph-mon[36481]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:12.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T08:30:12.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:12.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:12 smithi179 ceph-mon[36481]: Deploying daemon osd.2 on smithi179 2024-05-27T08:30:12.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:12 smithi092 ceph-mon[36049]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T08:30:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:12 smithi092 ceph-mon[36049]: Deploying daemon osd.2 on smithi179 2024-05-27T08:30:12.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:12 smithi077 ceph-mon[32007]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:12.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-27T08:30:12.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:12.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:12 smithi077 ceph-mon[32007]: Deploying daemon osd.2 on smithi179 2024-05-27T08:30:14.649 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:14 smithi179 ceph-mon[36481]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:14.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:14 smithi092 ceph-mon[36049]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:14.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:14 smithi077 ceph-mon[32007]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:16.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:16 smithi092 ceph-mon[36049]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:16.791 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:16 smithi179 ceph-mon[36481]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:16.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:16 smithi077 ceph-mon[32007]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:18.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:18 smithi179 ceph-mon[36481]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:18 smithi077 ceph-mon[32007]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:18 smithi092 ceph-mon[36049]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:20.284 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:20 smithi179 ceph-mon[36481]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:20.333 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 2 on host 'smithi179' 2024-05-27T08:30:20.333 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.331+0000 7fefc37fe700 1 -- 172.21.15.179:0/3274878686 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fefd406c3d0 con 0x7fefbc0b27f0 2024-05-27T08:30:20.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fefbc0b27f0 msgr2=0x7fefbc0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:20.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fefbc0b27f0 0x7fefbc0b4cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fefc400b5c0 tx=0x7fefc4005fb0 comp rx=0 tx=0).stop 2024-05-27T08:30:20.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41024b0 msgr2=0x7fefd4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:20.335 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41024b0 0x7fefd4199760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fefc800d9b0 tx=0x7fefc800dcc0 comp rx=0 tx=0).stop 2024-05-27T08:30:20.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 shutdown_connections 2024-05-27T08:30:20.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fefd4104050 0x7fefd41a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:20.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fefbc0b27f0 0x7fefbc0b4cb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:20.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefd41024b0 0x7fefd4199760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:20.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 --2- 172.21.15.179:0/3274878686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fefd41036b0 0x7fefd4199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:20.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 >> 172.21.15.179:0/3274878686 conn(0x7fefd40fda70 msgr2=0x7fefd406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:30:20.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 shutdown_connections 2024-05-27T08:30:20.336 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:30:20.334+0000 7fefd8e60700 1 -- 172.21.15.179:0/3274878686 wait complete. 2024-05-27T08:30:20.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:20 smithi092 ceph-mon[36049]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:20.595 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:20 smithi077 ceph-mon[32007]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:20.933 DEBUG:teuthology.orchestra.run.smithi179:osd.2> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.2.service 2024-05-27T08:30:20.936 INFO:tasks.cephadm:Deploying osd.3 on smithi077 with /dev/vg_nvme/lv_3... 2024-05-27T08:30:20.936 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T08:30:21.181 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:30:21.315 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:20 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2[41612]: 2024-05-27T08:30:20.968+0000 7fb6ef5437c0 -1 osd.2 0 log_to_monitors true 2024-05-27T08:30:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:21 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:21 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:21 smithi179 ceph-mon[36481]: from='osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T08:30:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:21 smithi179 ceph-mon[36481]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T08:30:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:21 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:21 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.614 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:21 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.614 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:21 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.614 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:21 smithi077 ceph-mon[32007]: from='osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T08:30:21.614 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:21 smithi077 ceph-mon[32007]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T08:30:21.615 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:21 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.615 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:21 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:21 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:21 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:21 smithi092 ceph-mon[36049]: from='osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T08:30:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:21 smithi092 ceph-mon[36049]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-27T08:30:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:21 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:21 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:22.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:22 smithi077 ceph-mon[32007]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:22.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:22 smithi077 ceph-mon[32007]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T08:30:22.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:22 smithi077 ceph-mon[32007]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T08:30:22.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:22.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:22 smithi077 ceph-mon[32007]: from='osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:30:22.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:22 smithi077 ceph-mon[32007]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:30:22.577 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:22 smithi179 ceph-mon[36481]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:22.578 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:22 smithi179 ceph-mon[36481]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T08:30:22.578 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:22 smithi179 ceph-mon[36481]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T08:30:22.578 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:22.578 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:22 smithi179 ceph-mon[36481]: from='osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:30:22.578 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:22 smithi179 ceph-mon[36481]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:30:22.578 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:22 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2[41612]: 2024-05-27T08:30:22.325+0000 7fb6e4ba7700 -1 osd.2 0 waiting for initial osdmap 2024-05-27T08:30:22.578 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:22 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2[41612]: 2024-05-27T08:30:22.330+0000 7fb6e099a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:30:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:22 smithi092 ceph-mon[36049]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:22 smithi092 ceph-mon[36049]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-27T08:30:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:22 smithi092 ceph-mon[36049]: osdmap e16: 3 total, 2 up, 3 in 2024-05-27T08:30:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:22 smithi092 ceph-mon[36049]: from='osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:30:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:22 smithi092 ceph-mon[36049]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:30:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T08:30:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T08:30:23.129 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-05-27T08:30:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-05-27T08:30:23.129 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128194 s, 818 MB/s 2024-05-27T08:30:23.129 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-05-27T08:30:23.501 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:23 smithi077 ceph-mon[32007]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:30:23.502 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:23 smithi077 ceph-mon[32007]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T08:30:23.502 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:23 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:23.502 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:23 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:23 smithi179 ceph-mon[36481]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:30:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:23 smithi179 ceph-mon[36481]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T08:30:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:23 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:23 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:23.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:23 smithi092 ceph-mon[36049]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:30:23.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:23 smithi092 ceph-mon[36049]: osdmap e17: 3 total, 2 up, 3 in 2024-05-27T08:30:23.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:23 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:23.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:23 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:23.813 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-05-27T08:30:24.050 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:30:24.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:30:24.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:30:24.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:24.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] boot 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:24.424 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:30:24.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] boot 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:30:24.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:30:24.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-27T08:30:24.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:24.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: osd.2 [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] boot 2024-05-27T08:30:24.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: osdmap e18: 3 total, 3 up, 3 in 2024-05-27T08:30:24.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:30:24.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:24.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:24.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:24.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:24.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:25.497 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:25 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi179 2024-05-27T08:30:25.497 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:25 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi179 to 17088M 2024-05-27T08:30:25.497 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:25 smithi077 ceph-mon[32007]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T08:30:25.497 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:25.497 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:25.498 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:25.498 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:25.498 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:30:25.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:25 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi179 2024-05-27T08:30:25.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:25 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi179 to 17088M 2024-05-27T08:30:25.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:25 smithi179 ceph-mon[36481]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T08:30:25.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:25 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:25.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:25 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:25.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:25 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:25.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:25 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:25.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:25 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:30:25.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:25 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi179 2024-05-27T08:30:25.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:25 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi179 to 17088M 2024-05-27T08:30:25.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:25 smithi092 ceph-mon[36049]: osdmap e19: 3 total, 3 up, 3 in 2024-05-27T08:30:25.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:25 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:25.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:25 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:25.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:25 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:25.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:25 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:25.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:25 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:30:25.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 -- 172.21.15.77:0/730374328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 msgr2=0x7fc4f41039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/730374328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 0x7fc4f41039e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0099c0 tx=0x7fc4dc009cd0 comp rx=0 tx=0).stop 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 -- 172.21.15.77:0/730374328 shutdown_connections 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/730374328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4f4105160 0x7fc4f410ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/730374328 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 0x7fc4f41039e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/730374328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f41047c0 0x7fc4f4104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 -- 172.21.15.77:0/730374328 >> 172.21.15.77:0/730374328 conn(0x7fc4f40feb60 msgr2=0x7fc4f4100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 -- 172.21.15.77:0/730374328 shutdown_connections 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.734+0000 7fc4fab2c700 1 -- 172.21.15.77:0/730374328 wait complete. 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4fab2c700 1 Processor -- start 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4fab2c700 1 -- start start 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4fab2c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 0x7fc4f40772e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4fab2c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4f41047c0 0x7fc4f4075930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4fab2c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f4105160 0x7fc4f4075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4fab2c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f410f050 con 0x7fc4f41035c0 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4fab2c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f410eed0 con 0x7fc4f41047c0 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4fab2c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f410f1d0 con 0x7fc4f4105160 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4f88c8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 0x7fc4f40772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4f88c8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 0x7fc4f40772e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:58990/0 (socket says 172.21.15.77:58990) 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4f88c8700 1 -- 172.21.15.77:0/1451302860 learned_addr learned my addr 172.21.15.77:0/1451302860 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.735+0000 7fc4f3fff700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4f41047c0 0x7fc4f4075930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f90c9700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f4105160 0x7fc4f4075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f88c8700 1 -- 172.21.15.77:0/1451302860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f4105160 msgr2=0x7fc4f4075e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f88c8700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f4105160 0x7fc4f4075e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f88c8700 1 -- 172.21.15.77:0/1451302860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4f41047c0 msgr2=0x7fc4f4075930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f88c8700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4f41047c0 0x7fc4f4075930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f88c8700 1 -- 172.21.15.77:0/1451302860 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f4076720 con 0x7fc4f41035c0 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f88c8700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 0x7fc4f40772e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0038c0 tx=0x7fc4dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f1ffb700 1 -- 172.21.15.77:0/1451302860 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc027070 con 0x7fc4f41035c0 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f1ffb700 1 -- 172.21.15.77:0/1451302860 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4dc004160 con 0x7fc4f41035c0 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f41a3850 con 0x7fc4f41035c0 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4f1ffb700 1 -- 172.21.15.77:0/1451302860 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc022e20 con 0x7fc4f41035c0 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.736+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f41a3d30 con 0x7fc4f41035c0 2024-05-27T08:30:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.737+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f406a230 con 0x7fc4f41035c0 2024-05-27T08:30:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.738+0000 7fc4f1ffb700 1 -- 172.21.15.77:0/1451302860 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fc4dc02a070 con 0x7fc4f41035c0 2024-05-27T08:30:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.739+0000 7fc4f1ffb700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc4e00b2870 0x7fc4e00b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.739+0000 7fc4f3fff700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc4e00b2870 0x7fc4e00b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.739+0000 7fc4f1ffb700 1 -- 172.21.15.77:0/1451302860 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7fc4dc0e1900 con 0x7fc4f41035c0 2024-05-27T08:30:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.740+0000 7fc4f3fff700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc4e00b2870 0x7fc4e00b4d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4003f90 tx=0x7fc4e4009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:30:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.743+0000 7fc4f1ffb700 1 -- 172.21.15.77:0/1451302860 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc4dc013070 con 0x7fc4f41035c0 2024-05-27T08:30:25.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:25.956+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc4f4063980 con 0x7fc4e00b2870 2024-05-27T08:30:26.805 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51271]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T08:30:26.805 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51271]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:26.805 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51271]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:26.805 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51271]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:26.805 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51274]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T08:30:26.805 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51274]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:26.805 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51274]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:26.805 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51274]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:26.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 ceph-mon[32007]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:26.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T08:30:26.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 ceph-mon[32007]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T08:30:26.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:30:26.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 ceph-mon[32007]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:26.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:26.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:26.806 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:26.856 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45517]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T08:30:26.856 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45517]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:26.856 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45517]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:26.856 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45517]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:26.857 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45520]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T08:30:26.857 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45520]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:26.857 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45520]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:26.857 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45520]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 ceph-mon[36481]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T08:30:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 ceph-mon[36481]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T08:30:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:30:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 ceph-mon[36481]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:26.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:26.899 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 ceph-mon[36049]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:26.899 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-27T08:30:26.900 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 ceph-mon[36049]: osdmap e20: 3 total, 3 up, 3 in 2024-05-27T08:30:26.900 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:30:26.900 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 ceph-mon[36049]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:26.900 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:26.900 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:26.900 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:26.900 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T08:30:26.900 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:26.900 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44762]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:26.900 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44762]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:26.901 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44765]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T08:30:26.901 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44765]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:26.901 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44765]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:26.901 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44765]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:27.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51286]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T08:30:27.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:27.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51286]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:27.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51286]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:27.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T08:30:27.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:27.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51299]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:27.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:26 smithi077 sudo[51299]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44768]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T08:30:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44768]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44768]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44768]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:27.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44771]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T08:30:27.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44771]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:27.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44771]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:27.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:26 smithi092 sudo[44771]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45523]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-27T08:30:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45523]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45523]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45523]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45526]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-27T08:30:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45526]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-27T08:30:27.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45526]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-27T08:30:27.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:26 smithi179 sudo[45526]: pam_unix(sudo:session): session closed for user root 2024-05-27T08:30:27.776 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T08:30:27.776 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T08:30:27.776 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.776 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.777 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.778 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:27 smithi077 ceph-mon[32007]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:27.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:27.818+0000 7fc4f1ffb700 1 -- 172.21.15.77:0/1451302860 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc4dc084840 con 0x7fc4f41035c0 2024-05-27T08:30:27.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T08:30:27.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T08:30:27.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:27 smithi179 ceph-mon[36481]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-27T08:30:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: osdmap e21: 3 total, 3 up, 3 in 2024-05-27T08:30:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:30:27.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:27 smithi092 ceph-mon[36049]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-27T08:30:28.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:28 smithi179 ceph-mon[36481]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:28.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:28 smithi179 ceph-mon[36481]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T08:30:28.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:28 smithi179 ceph-mon[36481]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-05-27T08:30:28.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:28 smithi077 ceph-mon[32007]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:28.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:28 smithi077 ceph-mon[32007]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T08:30:28.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:28 smithi077 ceph-mon[32007]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-05-27T08:30:28.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:28 smithi092 ceph-mon[36049]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:28.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:28 smithi092 ceph-mon[36049]: osdmap e22: 3 total, 3 up, 3 in 2024-05-27T08:30:28.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:28 smithi092 ceph-mon[36049]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-05-27T08:30:29.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:29 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2929466111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96e70a03-bba7-4d18-a3f4-d14c454f719f"}]: dispatch 2024-05-27T08:30:29.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:29 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2929466111' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96e70a03-bba7-4d18-a3f4-d14c454f719f"}]': finished 2024-05-27T08:30:29.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:29 smithi179 ceph-mon[36481]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T08:30:29.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:29 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:29.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:29 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1668553949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:29.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:29 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2929466111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96e70a03-bba7-4d18-a3f4-d14c454f719f"}]: dispatch 2024-05-27T08:30:29.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:29 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2929466111' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96e70a03-bba7-4d18-a3f4-d14c454f719f"}]': finished 2024-05-27T08:30:29.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:29 smithi077 ceph-mon[32007]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T08:30:29.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:29.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:29 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1668553949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:29.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:29 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2929466111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "96e70a03-bba7-4d18-a3f4-d14c454f719f"}]: dispatch 2024-05-27T08:30:29.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:29 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2929466111' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "96e70a03-bba7-4d18-a3f4-d14c454f719f"}]': finished 2024-05-27T08:30:29.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:29 smithi092 ceph-mon[36049]: osdmap e23: 4 total, 3 up, 4 in 2024-05-27T08:30:29.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:29 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:29.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:29 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1668553949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:30.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:30 smithi092 ceph-mon[36049]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:31.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:30 smithi179 ceph-mon[36481]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:31.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:30 smithi077 ceph-mon[32007]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:32.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:32 smithi077 ceph-mon[32007]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:32.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:32 smithi092 ceph-mon[36049]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:33.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:32 smithi179 ceph-mon[36481]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:34.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:34 smithi092 ceph-mon[36049]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:34.986 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:34 smithi077 ceph-mon[32007]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:35.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:34 smithi179 ceph-mon[36481]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:36.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:36 smithi077 ceph-mon[32007]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:36.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T08:30:36.941 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:36.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:36 smithi092 ceph-mon[36049]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T08:30:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:36 smithi179 ceph-mon[36481]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-27T08:30:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:37.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:37 smithi092 ceph-mon[36049]: Deploying daemon osd.3 on smithi077 2024-05-27T08:30:38.007 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:37 smithi179 ceph-mon[36481]: Deploying daemon osd.3 on smithi077 2024-05-27T08:30:38.018 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:37 smithi077 ceph-mon[32007]: Deploying daemon osd.3 on smithi077 2024-05-27T08:30:38.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:38 smithi092 ceph-mon[36049]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:39.082 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:38 smithi077 ceph-mon[32007]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:39.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:38 smithi179 ceph-mon[36481]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:40.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:40 smithi092 ceph-mon[36049]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:41.091 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:40 smithi077 ceph-mon[32007]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:41.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:40 smithi179 ceph-mon[36481]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:42.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:42 smithi179 ceph-mon[36481]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:42.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:42 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:42.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:42 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:42.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:42 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:42.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:42 smithi077 ceph-mon[32007]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:42.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:42 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:42.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:42 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:42.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:42 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:42.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:42 smithi092 ceph-mon[36049]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:42.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:42 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:42.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:42 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:42.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:42 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:44.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:44 smithi179 ceph-mon[36481]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:44.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:44 smithi077 ceph-mon[32007]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:44.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:44 smithi092 ceph-mon[36049]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:44.987 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 3 on host 'smithi077' 2024-05-27T08:30:44.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.985+0000 7fc4f1ffb700 1 -- 172.21.15.77:0/1451302860 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc4f4063980 con 0x7fc4e00b2870 2024-05-27T08:30:44.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc4e00b2870 msgr2=0x7fc4e00b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:44.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc4e00b2870 0x7fc4e00b4d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4003f90 tx=0x7fc4e4009180 comp rx=0 tx=0).stop 2024-05-27T08:30:44.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 msgr2=0x7fc4f40772e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 0x7fc4f40772e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc0038c0 tx=0x7fc4dc0047b0 comp rx=0 tx=0).stop 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 shutdown_connections 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc4e00b2870 0x7fc4e00b4d30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4f4105160 0x7fc4f4075e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4f41035c0 0x7fc4f40772e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 --2- 172.21.15.77:0/1451302860 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4f41047c0 0x7fc4f4075930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 >> 172.21.15.77:0/1451302860 conn(0x7fc4f40feb60 msgr2=0x7fc4f4107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 shutdown_connections 2024-05-27T08:30:44.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:30:44.989+0000 7fc4fab2c700 1 -- 172.21.15.77:0/1451302860 wait complete. 2024-05-27T08:30:45.617 DEBUG:teuthology.orchestra.run.smithi077:osd.3> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.3.service 2024-05-27T08:30:45.620 INFO:tasks.cephadm:Deploying osd.4 on smithi092 with /dev/vg_nvme/lv_3... 2024-05-27T08:30:45.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T08:30:45.860 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:30:46.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:45 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:45 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:45 smithi092 ceph-mon[36049]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T08:30:46.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:45 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:45 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:45 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:45 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:45 smithi179 ceph-mon[36481]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T08:30:46.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:45 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:45 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:45 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:45 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:45 smithi077 ceph-mon[32007]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-27T08:30:46.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:45 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:46.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:45 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:47.044 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:47 smithi092 ceph-mon[36049]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:47.044 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:47 smithi092 ceph-mon[36049]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T08:30:47.045 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:47 smithi092 ceph-mon[36049]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T08:30:47.045 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:47 smithi092 ceph-mon[36049]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:30:47.045 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:47 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:47.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:47 smithi179 ceph-mon[36481]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:47.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:47 smithi179 ceph-mon[36481]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T08:30:47.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:47 smithi179 ceph-mon[36481]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T08:30:47.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:47 smithi179 ceph-mon[36481]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:30:47.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:47 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:47.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:47 smithi077 ceph-mon[32007]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:47.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:47 smithi077 ceph-mon[32007]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-27T08:30:47.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:47 smithi077 ceph-mon[32007]: osdmap e24: 4 total, 3 up, 4 in 2024-05-27T08:30:47.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:47 smithi077 ceph-mon[32007]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:30:47.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:47 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:47.593 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T08:30:47.593 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T08:30:47.593 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-27T08:30:47.593 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-27T08:30:47.594 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113311 s, 925 MB/s 2024-05-27T08:30:47.594 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-27T08:30:48.029 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:30:47 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3[53416]: 2024-05-27T08:30:47.930+0000 7fe5772f1700 -1 osd.3 0 waiting for initial osdmap 2024-05-27T08:30:48.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_3 2024-05-27T08:30:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:30:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T08:30:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' 2024-05-27T08:30:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: osdmap e26: 4 total, 3 up, 4 in 2024-05-27T08:30:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.379 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:30:48 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3[53416]: 2024-05-27T08:30:48.125+0000 7fe56f0dc700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:30:48.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:30:48.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T08:30:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' 2024-05-27T08:30:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: osdmap e26: 4 total, 3 up, 4 in 2024-05-27T08:30:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.421 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: osdmap e25: 4 total, 3 up, 4 in 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: from='osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768]' entity='osd.3' 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: osdmap e26: 4 total, 3 up, 4 in 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:48.461 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:49.281 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:49 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:30:49.281 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:49 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:30:49.281 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:49 smithi179 ceph-mon[36481]: osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] boot 2024-05-27T08:30:49.281 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:49 smithi179 ceph-mon[36481]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T08:30:49.281 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:49 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:49.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:49 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:30:49.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:49 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:30:49.312 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:49 smithi092 ceph-mon[36049]: osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] boot 2024-05-27T08:30:49.312 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:49 smithi092 ceph-mon[36049]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T08:30:49.312 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:49 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:49.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:49 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:30:49.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:49 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:30:49.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:49 smithi077 ceph-mon[32007]: osd.3 [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] boot 2024-05-27T08:30:49.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:49 smithi077 ceph-mon[32007]: osdmap e27: 4 total, 4 up, 4 in 2024-05-27T08:30:49.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:49 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:30:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 -- 172.21.15.92:0/2111224219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e80104040 msgr2=0x7f0e8010a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 --2- 172.21.15.92:0/2111224219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e80104040 0x7f0e8010a8f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b0d0 tx=0x7f0e7800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:30:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 -- 172.21.15.92:0/2111224219 shutdown_connections 2024-05-27T08:30:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 --2- 172.21.15.92:0/2111224219 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e80104040 0x7f0e8010a8f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 --2- 172.21.15.92:0/2111224219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e801024a0 0x7f0e801028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 --2- 172.21.15.92:0/2111224219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e801036a0 0x7f0e80103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 -- 172.21.15.92:0/2111224219 >> 172.21.15.92:0/2111224219 conn(0x7f0e800fda40 msgr2=0x7f0e800ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:30:50.140 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 -- 172.21.15.92:0/2111224219 shutdown_connections 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.124+0000 7f0e882f6700 1 -- 172.21.15.92:0/2111224219 wait complete. 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e882f6700 1 Processor -- start 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e882f6700 1 -- start start 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e882f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e801024a0 0x7f0e80199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e882f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e801036a0 0x7f0e80199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e882f6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e80104040 0x7f0e801a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e882f6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8010dfc0 con 0x7f0e80104040 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e882f6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8010de40 con 0x7f0e801036a0 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e882f6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8010e140 con 0x7f0e801024a0 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e85891700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e801036a0 0x7f0e80199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e85891700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e801036a0 0x7f0e80199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36214/0 (socket says 172.21.15.92:36214) 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e85891700 1 -- 172.21.15.92:0/548796370 learned_addr learned my addr 172.21.15.92:0/548796370 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e86092700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e801024a0 0x7f0e80199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:50.141 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e86893700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e80104040 0x7f0e801a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e85891700 1 -- 172.21.15.92:0/548796370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e801024a0 msgr2=0x7f0e80199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e85891700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e801024a0 0x7f0e80199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e85891700 1 -- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e80104040 msgr2=0x7f0e801a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e85891700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e80104040 0x7f0e801a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e85891700 1 -- 172.21.15.92:0/548796370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e801a13d0 con 0x7f0e801036a0 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.125+0000 7f0e86092700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e801024a0 0x7f0e80199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.126+0000 7f0e86893700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e80104040 0x7f0e801a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.126+0000 7f0e85891700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e801036a0 0x7f0e80199cb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00d980 tx=0x7f0e7c00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.126+0000 7f0e777fe700 1 -- 172.21.15.92:0/548796370 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c016070 con 0x7f0e801036a0 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.126+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e801a16c0 con 0x7f0e801036a0 2024-05-27T08:30:50.142 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.126+0000 7f0e777fe700 1 -- 172.21.15.92:0/548796370 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7c0044d0 con 0x7f0e801036a0 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.126+0000 7f0e777fe700 1 -- 172.21.15.92:0/548796370 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c004cf0 con 0x7f0e801036a0 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.126+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e801a1c00 con 0x7f0e801036a0 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.127+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8006a230 con 0x7f0e801036a0 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.128+0000 7f0e777fe700 1 -- 172.21.15.92:0/548796370 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0e7c004020 con 0x7f0e801036a0 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.129+0000 7f0e777fe700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.129+0000 7f0e86092700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.129+0000 7f0e777fe700 1 -- 172.21.15.92:0/548796370 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f0e7c0e0f40 con 0x7f0e801036a0 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.130+0000 7f0e86092700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e80103500 tx=0x7f0e70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:30:50.143 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.134+0000 7f0e777fe700 1 -- 172.21.15.92:0/548796370 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e7c01f020 con 0x7f0e801036a0 2024-05-27T08:30:50.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:30:50.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi077 2024-05-27T08:30:50.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:50.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:50.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi077 to 8544M 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:50 smithi092 ceph-mon[36049]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T08:30:50.354 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:30:50.353+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0e800ffc60 con 0x7f0e6c0b2880 2024-05-27T08:30:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:30:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi077 2024-05-27T08:30:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi077 to 8544M 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:50 smithi179 ceph-mon[36481]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T08:30:50.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:30:50.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi077 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi077 to 8544M 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:30:50.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:30:50.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:50 smithi077 ceph-mon[32007]: osdmap e28: 4 total, 4 up, 4 in 2024-05-27T08:30:51.204 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:51 smithi092 ceph-mon[36049]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:51.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:51.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:51.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:51.205 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:51 smithi092 ceph-mon[36049]: osdmap e29: 4 total, 4 up, 4 in 2024-05-27T08:30:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:51 smithi179 ceph-mon[36481]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:51.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:51 smithi179 ceph-mon[36481]: osdmap e29: 4 total, 4 up, 4 in 2024-05-27T08:30:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:51 smithi077 ceph-mon[32007]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:30:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:30:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:30:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:30:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:51 smithi077 ceph-mon[32007]: osdmap e29: 4 total, 4 up, 4 in 2024-05-27T08:30:53.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:52 smithi092 ceph-mon[36049]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:30:53.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:52 smithi179 ceph-mon[36481]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:30:53.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:52 smithi077 ceph-mon[32007]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:30:54.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:54 smithi179 ceph-mon[36481]: from='client.? 172.21.15.92:0/525938959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]: dispatch 2024-05-27T08:30:54.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:54 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]: dispatch 2024-05-27T08:30:54.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:54 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]': finished 2024-05-27T08:30:54.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:54 smithi179 ceph-mon[36481]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T08:30:54.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:30:54.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:54 smithi179 ceph-mon[36481]: from='client.? 172.21.15.92:0/1528311356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:54.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:54 smithi077 ceph-mon[32007]: from='client.? 172.21.15.92:0/525938959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]: dispatch 2024-05-27T08:30:54.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:54 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]: dispatch 2024-05-27T08:30:54.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:54 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]': finished 2024-05-27T08:30:54.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:54 smithi077 ceph-mon[32007]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T08:30:54.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:30:54.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:54 smithi077 ceph-mon[32007]: from='client.? 172.21.15.92:0/1528311356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:54.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:54 smithi092 ceph-mon[36049]: from='client.? 172.21.15.92:0/525938959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]: dispatch 2024-05-27T08:30:54.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:54 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]: dispatch 2024-05-27T08:30:54.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:54 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d93ff380-9258-4203-9826-6ddedfa1d03b"}]': finished 2024-05-27T08:30:54.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:54 smithi092 ceph-mon[36049]: osdmap e30: 5 total, 4 up, 5 in 2024-05-27T08:30:54.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:30:54.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:54 smithi092 ceph-mon[36049]: from='client.? 172.21.15.92:0/1528311356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:30:55.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:55 smithi179 ceph-mon[36481]: pgmap v108: 1 pgs: 1 activating+degraded; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T08:30:55.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:55 smithi179 ceph-mon[36481]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T08:30:55.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:55 smithi077 ceph-mon[32007]: pgmap v108: 1 pgs: 1 activating+degraded; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T08:30:55.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:55 smithi077 ceph-mon[32007]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T08:30:55.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:55 smithi092 ceph-mon[36049]: pgmap v108: 1 pgs: 1 activating+degraded; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T08:30:55.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:55 smithi092 ceph-mon[36049]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-27T08:30:56.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:56 smithi179 ceph-mon[36481]: pgmap v109: 1 pgs: 1 activating+degraded; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T08:30:56.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:56 smithi077 ceph-mon[32007]: pgmap v109: 1 pgs: 1 activating+degraded; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T08:30:56.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:56 smithi092 ceph-mon[36049]: pgmap v109: 1 pgs: 1 activating+degraded; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-27T08:30:57.470 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:57 smithi092 ceph-mon[36049]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T08:30:57.470 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:57 smithi092 ceph-mon[36049]: Cluster is now healthy 2024-05-27T08:30:57.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:57 smithi179 ceph-mon[36481]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T08:30:57.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:57 smithi179 ceph-mon[36481]: Cluster is now healthy 2024-05-27T08:30:57.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:57 smithi077 ceph-mon[32007]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-27T08:30:57.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:57 smithi077 ceph-mon[32007]: Cluster is now healthy 2024-05-27T08:30:58.423 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:30:58 smithi092 ceph-mon[36049]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-27T08:30:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:30:58 smithi179 ceph-mon[36481]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-27T08:30:58.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:30:58 smithi077 ceph-mon[32007]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-05-27T08:31:00.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:00 smithi092 ceph-mon[36049]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-27T08:31:00.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:00 smithi179 ceph-mon[36481]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-27T08:31:00.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:00 smithi077 ceph-mon[32007]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-05-27T08:31:01.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T08:31:01.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:01.311 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:01 smithi092 ceph-mon[36049]: Deploying daemon osd.4 on smithi092 2024-05-27T08:31:01.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T08:31:01.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:01.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:01 smithi179 ceph-mon[36481]: Deploying daemon osd.4 on smithi092 2024-05-27T08:31:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-27T08:31:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:01 smithi077 ceph-mon[32007]: Deploying daemon osd.4 on smithi092 2024-05-27T08:31:02.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:02 smithi092 ceph-mon[36049]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:31:02.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:02 smithi179 ceph-mon[36481]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:31:02.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:02 smithi077 ceph-mon[32007]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:31:04.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:04 smithi092 ceph-mon[36049]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-27T08:31:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:04 smithi179 ceph-mon[36481]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-27T08:31:04.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:04 smithi077 ceph-mon[32007]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-27T08:31:06.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:06 smithi077 ceph-mon[32007]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-27T08:31:06.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:06 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:06.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:06 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:06.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:06 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:06.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:06 smithi092 ceph-mon[36049]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-27T08:31:06.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:06 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:06.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:06 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:06.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:06 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:06 smithi179 ceph-mon[36481]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-27T08:31:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:06 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:06 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:06 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:08.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:08 smithi092 ceph-mon[36049]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-27T08:31:08.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:08.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:08.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:08.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:08.666 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:08.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:08 smithi077 ceph-mon[32007]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-27T08:31:08.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:08.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:08.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:08.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:08.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:08.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:08 smithi179 ceph-mon[36481]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-27T08:31:08.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:08.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:08.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:08.909 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:08.909 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:09.519 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.518+0000 7f0e777fe700 1 -- 172.21.15.92:0/548796370 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e800ffc60 con 0x7f0e6c0b2880 2024-05-27T08:31:09.520 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 4 on host 'smithi092' 2024-05-27T08:31:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0e6c0b2880 msgr2=0x7f0e6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e80103500 tx=0x7f0e70008040 comp rx=0 tx=0).stop 2024-05-27T08:31:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e801036a0 msgr2=0x7f0e80199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:09.522 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e801036a0 0x7f0e80199cb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00d980 tx=0x7f0e7c00dd40 comp rx=0 tx=0).stop 2024-05-27T08:31:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 shutdown_connections 2024-05-27T08:31:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e80104040 0x7f0e801a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e801024a0 0x7f0e80199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 --2- 172.21.15.92:0/548796370 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0e801036a0 0x7f0e80199cb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 >> 172.21.15.92:0/548796370 conn(0x7f0e800fda40 msgr2=0x7f0e800ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:31:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 shutdown_connections 2024-05-27T08:31:09.523 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:31:09.521+0000 7f0e882f6700 1 -- 172.21.15.92:0/548796370 wait complete. 2024-05-27T08:31:10.133 DEBUG:teuthology.orchestra.run.smithi092:osd.4> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.4.service 2024-05-27T08:31:10.135 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2024-05-27T08:31:10.136 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-27T08:31:10.389 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:31:10.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:10 smithi092 ceph-mon[36049]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:10.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:10 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:10.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:10 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:10.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:10 smithi092 ceph-mon[36049]: from='osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T08:31:10.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:10 smithi092 ceph-mon[36049]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T08:31:10.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:10 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:10.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:10 smithi179 ceph-mon[36481]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:10.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:10 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:10.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:10 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:10.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:10 smithi179 ceph-mon[36481]: from='osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T08:31:10.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:10 smithi179 ceph-mon[36481]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T08:31:10.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:10 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:10.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:10 smithi077 ceph-mon[32007]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:10.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:10 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:10.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:10 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:10.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:10 smithi077 ceph-mon[32007]: from='osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T08:31:10.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:10 smithi077 ceph-mon[32007]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-27T08:31:10.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:10 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:11.571 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:11 smithi092 ceph-mon[36049]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T08:31:11.572 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:11 smithi092 ceph-mon[36049]: from='osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:31:11.572 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:11 smithi092 ceph-mon[36049]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T08:31:11.572 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:11.572 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:11 smithi092 ceph-mon[36049]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:31:11.572 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:31:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4[47571]: 2024-05-27T08:31:11.478+0000 7faae9cd6700 -1 osd.4 0 waiting for initial osdmap 2024-05-27T08:31:11.572 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:31:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4[47571]: 2024-05-27T08:31:11.486+0000 7faae5ac9700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:31:11.781 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:11 smithi179 ceph-mon[36481]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T08:31:11.781 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:11 smithi179 ceph-mon[36481]: from='osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:31:11.781 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:11 smithi179 ceph-mon[36481]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T08:31:11.781 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:11.782 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:11 smithi179 ceph-mon[36481]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:31:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:11 smithi077 ceph-mon[32007]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-27T08:31:11.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:11 smithi077 ceph-mon[32007]: from='osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:31:11.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:11 smithi077 ceph-mon[32007]: osdmap e31: 5 total, 4 up, 5 in 2024-05-27T08:31:11.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:11.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:11 smithi077 ceph-mon[32007]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:31:12.167 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-27T08:31:12.168 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-27T08:31:12.168 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-05-27T08:31:12.168 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-05-27T08:31:12.168 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126068 s, 832 MB/s 2024-05-27T08:31:12.168 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-05-27T08:31:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:12 smithi092 ceph-mon[36049]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:12 smithi092 ceph-mon[36049]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:31:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:12 smithi092 ceph-mon[36049]: osdmap e32: 5 total, 4 up, 5 in 2024-05-27T08:31:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:12 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:12 smithi179 ceph-mon[36481]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:12.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:12 smithi179 ceph-mon[36481]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:31:12.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:12 smithi179 ceph-mon[36481]: osdmap e32: 5 total, 4 up, 5 in 2024-05-27T08:31:12.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:12 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.762 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-05-27T08:31:12.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:12 smithi077 ceph-mon[32007]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:12.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:12 smithi077 ceph-mon[32007]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:31:12.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:12 smithi077 ceph-mon[32007]: osdmap e32: 5 total, 4 up, 5 in 2024-05-27T08:31:12.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:12 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:12.995 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:31:13.741 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:31:13.741 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] boot 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi092 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:13.742 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:13.743 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:13 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:31:13.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:31:13.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] boot 2024-05-27T08:31:13.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi092 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:13.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:13 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:31:13.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:31:13.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: osd.4 [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] boot 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: osdmap e33: 5 total, 5 up, 5 in 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi092 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi092 to 8544M 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:13.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:13 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:14.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 -- 172.21.15.179:0/4117395869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 msgr2=0x7f33c4103ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:14.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/4117395869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 0x7f33c4103ea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33b40099c0 tx=0x7f33b4009cd0 comp rx=0 tx=0).stop 2024-05-27T08:31:14.540 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 -- 172.21.15.179:0/4117395869 shutdown_connections 2024-05-27T08:31:14.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/4117395869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c4103160 0x7f33c410aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:14.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/4117395869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 0x7f33c4103ea0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:14.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/4117395869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c41027c0 0x7f33c4102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:14.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 -- 172.21.15.179:0/4117395869 >> 172.21.15.179:0/4117395869 conn(0x7f33c40fd210 msgr2=0x7f33c40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:31:14.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 -- 172.21.15.179:0/4117395869 shutdown_connections 2024-05-27T08:31:14.541 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.538+0000 7f33c8e0b700 1 -- 172.21.15.179:0/4117395869 wait complete. 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c8e0b700 1 Processor -- start 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c8e0b700 1 -- start start 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c8e0b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c41027c0 0x7f33c4106560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c8e0b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c4103160 0x7f33c4104bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c8e0b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 0x7f33c41050f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c8e0b700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33c410dff0 con 0x7f33c4103160 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c8e0b700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f33c410de70 con 0x7f33c41027c0 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c8e0b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33c410e170 con 0x7f33c4103a80 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.539+0000 7f33c2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 0x7f33c41050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c2d9d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 0x7f33c41050f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:47992/0 (socket says 172.21.15.179:47992) 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c2d9d700 1 -- 172.21.15.179:0/3715226467 learned_addr learned my addr 172.21.15.179:0/3715226467 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:31:14.543 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c1d9b700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c4103160 0x7f33c4104bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c259c700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c41027c0 0x7f33c4106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c2d9d700 1 -- 172.21.15.179:0/3715226467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c41027c0 msgr2=0x7f33c4106560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c2d9d700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c41027c0 0x7f33c4106560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c2d9d700 1 -- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c4103160 msgr2=0x7f33c4104bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c2d9d700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c4103160 0x7f33c4104bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c2d9d700 1 -- 172.21.15.179:0/3715226467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c41059a0 con 0x7f33c4103a80 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c2d9d700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 0x7f33c41050f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00b970 tx=0x7f33bc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33bb7fe700 1 -- 172.21.15.179:0/3715226467 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00d860 con 0x7f33c4103a80 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c41a3830 con 0x7f33c4103a80 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33bb7fe700 1 -- 172.21.15.179:0/3715226467 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33bc00e430 con 0x7f33c4103a80 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33bb7fe700 1 -- 172.21.15.179:0/3715226467 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc017660 con 0x7f33c4103a80 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.540+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c41a3d70 con 0x7f33c4103a80 2024-05-27T08:31:14.544 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.541+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c406a230 con 0x7f33c4103a80 2024-05-27T08:31:14.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.543+0000 7f33bb7fe700 1 -- 172.21.15.179:0/3715226467 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f33bc00da00 con 0x7f33c4103a80 2024-05-27T08:31:14.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.544+0000 7f33bb7fe700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f33b00b2880 0x7f33b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:31:14.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.544+0000 7f33bb7fe700 1 -- 172.21.15.179:0/3715226467 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f33bc0e1180 con 0x7f33c4103a80 2024-05-27T08:31:14.545 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.544+0000 7f33c259c700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f33b00b2880 0x7f33b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:31:14.546 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.545+0000 7f33c259c700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f33b00b2880 0x7f33b00b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f33b40038c0 tx=0x7f33b400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:31:14.549 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.548+0000 7f33bb7fe700 1 -- 172.21.15.179:0/3715226467 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f33bc013030 con 0x7f33c4103a80 2024-05-27T08:31:14.760 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:14.759+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f33c41a4090 con 0x7f33b00b2880 2024-05-27T08:31:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:14 smithi179 ceph-mon[36481]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:14 smithi179 ceph-mon[36481]: osdmap e34: 5 total, 5 up, 5 in 2024-05-27T08:31:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:31:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:31:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:14 smithi092 ceph-mon[36049]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:15.258 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:14 smithi092 ceph-mon[36049]: osdmap e34: 5 total, 5 up, 5 in 2024-05-27T08:31:15.259 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:31:15.259 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:31:15.259 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:15.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:14 smithi077 ceph-mon[32007]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-05-27T08:31:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:14 smithi077 ceph-mon[32007]: osdmap e34: 5 total, 5 up, 5 in 2024-05-27T08:31:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:31:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:31:15.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:16.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:15 smithi092 ceph-mon[36049]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:31:16.283 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:15 smithi179 ceph-mon[36481]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:31:16.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:15 smithi077 ceph-mon[32007]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:31:17.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:16 smithi092 ceph-mon[36049]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:17.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:16 smithi179 ceph-mon[36481]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:17.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:16 smithi077 ceph-mon[32007]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:17 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/2300690901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]: dispatch 2024-05-27T08:31:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:17 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]: dispatch 2024-05-27T08:31:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:17 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]': finished 2024-05-27T08:31:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:17 smithi179 ceph-mon[36481]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T08:31:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:17 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:18.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:17 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/2300690901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]: dispatch 2024-05-27T08:31:18.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:17 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]: dispatch 2024-05-27T08:31:18.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:17 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]': finished 2024-05-27T08:31:18.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:17 smithi092 ceph-mon[36049]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T08:31:18.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:17 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:18.280 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:17 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/2300690901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]: dispatch 2024-05-27T08:31:18.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:17 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]: dispatch 2024-05-27T08:31:18.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:17 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c05fe340-efa9-43dd-ac86-7d0f436df4a2"}]': finished 2024-05-27T08:31:18.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:17 smithi077 ceph-mon[32007]: osdmap e35: 6 total, 5 up, 6 in 2024-05-27T08:31:18.281 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:17 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:19.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:18 smithi092 ceph-mon[36049]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:19.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:18 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/2067325537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:31:19.300 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:18 smithi077 ceph-mon[32007]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:19.300 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:18 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/2067325537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:31:19.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:18 smithi179 ceph-mon[36481]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:19.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:18 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/2067325537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:31:21.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:20 smithi092 ceph-mon[36049]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:21.307 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:20 smithi179 ceph-mon[36481]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:21.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:20 smithi077 ceph-mon[32007]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:22 smithi179 ceph-mon[36481]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:22 smithi092 ceph-mon[36049]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:23.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:22 smithi077 ceph-mon[32007]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:24 smithi179 ceph-mon[36481]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T08:31:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:25.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:24 smithi092 ceph-mon[36049]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:25.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T08:31:25.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:25.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:24 smithi077 ceph-mon[32007]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:25.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-27T08:31:25.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:26.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:25 smithi092 ceph-mon[36049]: Deploying daemon osd.5 on smithi179 2024-05-27T08:31:26.243 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:25 smithi179 ceph-mon[36481]: Deploying daemon osd.5 on smithi179 2024-05-27T08:31:26.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:25 smithi077 ceph-mon[32007]: Deploying daemon osd.5 on smithi179 2024-05-27T08:31:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:26 smithi092 ceph-mon[36049]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:26 smithi179 ceph-mon[36481]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:27.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:26 smithi077 ceph-mon[32007]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:29.232 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:28 smithi179 ceph-mon[36481]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:29.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:28 smithi092 ceph-mon[36049]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:29.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:28 smithi077 ceph-mon[32007]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:31.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:30 smithi179 ceph-mon[36481]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:31.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:30 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:31.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:30 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:31.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:30 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:31.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:30 smithi092 ceph-mon[36049]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:31.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:30 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:31.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:30 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:31.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:30 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:31.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:30 smithi077 ceph-mon[32007]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:31.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:30 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:31.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:30 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:31.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:30 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:32 smithi092 ceph-mon[36049]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:33.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:32 smithi179 ceph-mon[36481]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:33.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.321+0000 7f33bb7fe700 1 -- 172.21.15.179:0/3715226467 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f33c41a4090 con 0x7f33b00b2880 2024-05-27T08:31:33.323 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-05-27T08:31:33.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f33b00b2880 msgr2=0x7f33b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:33.325 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f33b00b2880 0x7f33b00b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f33b40038c0 tx=0x7f33b400b540 comp rx=0 tx=0).stop 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 msgr2=0x7f33c41050f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 0x7f33c41050f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00b970 tx=0x7f33bc00bc80 comp rx=0 tx=0).stop 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 shutdown_connections 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f33b00b2880 0x7f33b00b4d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c4103a80 0x7f33c41050f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f33c41027c0 0x7f33c4106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 --2- 172.21.15.179:0/3715226467 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33c4103160 0x7f33c4104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 >> 172.21.15.179:0/3715226467 conn(0x7f33c40fd210 msgr2=0x7f33c40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 shutdown_connections 2024-05-27T08:31:33.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:31:33.324+0000 7f33c8e0b700 1 -- 172.21.15.179:0/3715226467 wait complete. 2024-05-27T08:31:33.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:32 smithi077 ceph-mon[32007]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:33.952 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.5.service 2024-05-27T08:31:33.954 INFO:tasks.cephadm:Deploying osd.6 on smithi077 with /dev/vg_nvme/lv_2... 2024-05-27T08:31:33.955 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T08:31:34.204 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:31:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:34 smithi179 ceph-mon[36481]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:34 smithi179 ceph-mon[36481]: from='osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T08:31:34.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:34 smithi179 ceph-mon[36481]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T08:31:34.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:34 smithi077 ceph-mon[32007]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:34.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:34 smithi077 ceph-mon[32007]: from='osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T08:31:34.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:34 smithi077 ceph-mon[32007]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T08:31:34.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:34 smithi092 ceph-mon[36049]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:34.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:34.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:34 smithi092 ceph-mon[36049]: from='osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T08:31:34.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:34 smithi092 ceph-mon[36049]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-27T08:31:35.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:35 smithi179 ceph-mon[36481]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T08:31:35.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:35 smithi179 ceph-mon[36481]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T08:31:35.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:35 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:35.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:35 smithi179 ceph-mon[36481]: from='osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:31:35.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:35 smithi179 ceph-mon[36481]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:31:35.702 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:35 smithi077 ceph-mon[32007]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T08:31:35.703 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:35 smithi077 ceph-mon[32007]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T08:31:35.703 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:35 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:35.703 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:35 smithi077 ceph-mon[32007]: from='osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:31:35.703 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:35 smithi077 ceph-mon[32007]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:31:35.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:35 smithi092 ceph-mon[36049]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-27T08:31:35.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:35 smithi092 ceph-mon[36049]: osdmap e36: 6 total, 5 up, 6 in 2024-05-27T08:31:35.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:35 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:35.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:35 smithi092 ceph-mon[36049]: from='osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:31:35.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:35 smithi092 ceph-mon[36049]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:31:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T08:31:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T08:31:36.277 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-05-27T08:31:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-05-27T08:31:36.277 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145301 s, 722 MB/s 2024-05-27T08:31:36.277 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-05-27T08:31:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:36 smithi179 ceph-mon[36481]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:36 smithi179 ceph-mon[36481]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:31:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:36 smithi179 ceph-mon[36481]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T08:31:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:36 smithi179 ceph-mon[36481]: osdmap e38: 6 total, 5 up, 6 in 2024-05-27T08:31:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:36 smithi179 ceph-mon[36481]: from='osd.5 ' entity='osd.5' 2024-05-27T08:31:36.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:31:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:31:36.296+0000 7f583042a700 -1 osd.5 0 waiting for initial osdmap 2024-05-27T08:31:36.480 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:31:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:31:36.477+0000 7f5827a14700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:31:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:36 smithi077 ceph-mon[32007]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:36 smithi077 ceph-mon[32007]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:31:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:36 smithi077 ceph-mon[32007]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T08:31:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:36 smithi077 ceph-mon[32007]: osdmap e38: 6 total, 5 up, 6 in 2024-05-27T08:31:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:36 smithi077 ceph-mon[32007]: from='osd.5 ' entity='osd.5' 2024-05-27T08:31:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:36 smithi092 ceph-mon[36049]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:36 smithi092 ceph-mon[36049]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:31:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:36 smithi092 ceph-mon[36049]: osdmap e37: 6 total, 5 up, 6 in 2024-05-27T08:31:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:36 smithi092 ceph-mon[36049]: osdmap e38: 6 total, 5 up, 6 in 2024-05-27T08:31:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:36 smithi092 ceph-mon[36049]: from='osd.5 ' entity='osd.5' 2024-05-27T08:31:36.985 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-05-27T08:31:37.242 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:31:37.618 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: osdmap e39: 6 total, 5 up, 6 in 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi179 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:37.619 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi179 to 8544M 2024-05-27T08:31:37.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:37.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:37.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:37.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:37.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:37.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:31:37.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:31:37.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: osdmap e39: 6 total, 5 up, 6 in 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi179 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi179 to 8544M 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:37.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:37.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:37.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: osdmap e39: 6 total, 5 up, 6 in 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi179 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:37.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi179 to 8544M 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:37.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:38.684 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:38 smithi077 ceph-mon[32007]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:38.684 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:38.684 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:38 smithi077 ceph-mon[32007]: osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] boot 2024-05-27T08:31:38.684 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:38 smithi077 ceph-mon[32007]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T08:31:38.684 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:38.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:38 smithi092 ceph-mon[36049]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:38.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:38.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:38 smithi092 ceph-mon[36049]: osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] boot 2024-05-27T08:31:38.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:38 smithi092 ceph-mon[36049]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T08:31:38.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:38.759 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:38 smithi179 ceph-mon[36481]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-05-27T08:31:38.759 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:38.759 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:38 smithi179 ceph-mon[36481]: osd.5 [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] boot 2024-05-27T08:31:38.759 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:38 smithi179 ceph-mon[36481]: osdmap e40: 6 total, 6 up, 6 in 2024-05-27T08:31:38.759 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:31:38.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 -- 172.21.15.77:0/178987469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 msgr2=0x7f8cd8103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/178987469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 0x7f8cd8103b10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00099c0 tx=0x7f8cc0009cd0 comp rx=0 tx=0).stop 2024-05-27T08:31:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 -- 172.21.15.77:0/178987469 shutdown_connections 2024-05-27T08:31:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/178987469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cd8104050 0x7f8cd810a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/178987469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81024b0 0x7f8cd81028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/178987469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 0x7f8cd8103b10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 -- 172.21.15.77:0/178987469 >> 172.21.15.77:0/178987469 conn(0x7f8cd80fda70 msgr2=0x7f8cd80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:31:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 -- 172.21.15.77:0/178987469 shutdown_connections 2024-05-27T08:31:38.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.889+0000 7f8cdd41a700 1 -- 172.21.15.77:0/178987469 wait complete. 2024-05-27T08:31:38.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cdd41a700 1 Processor -- start 2024-05-27T08:31:38.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cdd41a700 1 -- start start 2024-05-27T08:31:38.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cdd41a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81024b0 0x7f8cd81997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:31:38.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cdd41a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 0x7f8cd8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:31:38.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cdd41a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cd8104050 0x7f8cd81a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:31:38.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cdd41a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810dff0 con 0x7f8cd81036b0 2024-05-27T08:31:38.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cdd41a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810de70 con 0x7f8cd81024b0 2024-05-27T08:31:38.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cdd41a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd810e170 con 0x7f8cd8104050 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8ccffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 0x7f8cd8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8ccffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 0x7f8cd8199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42206/0 (socket says 172.21.15.77:42206) 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8ccffff700 1 -- 172.21.15.77:0/2000992959 learned_addr learned my addr 172.21.15.77:0/2000992959 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.890+0000 7f8cd77fe700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cd8104050 0x7f8cd81a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8cd6ffd700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81024b0 0x7f8cd81997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8ccffff700 1 -- 172.21.15.77:0/2000992959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cd8104050 msgr2=0x7f8cd81a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8ccffff700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cd8104050 0x7f8cd81a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8ccffff700 1 -- 172.21.15.77:0/2000992959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81024b0 msgr2=0x7f8cd81997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8ccffff700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81024b0 0x7f8cd81997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8ccffff700 1 -- 172.21.15.77:0/2000992959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd81a1450 con 0x7f8cd81036b0 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8cd6ffd700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81024b0 0x7f8cd81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8ccffff700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 0x7f8cd8199d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0000c00 tx=0x7f8cc0005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:31:38.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8cd4ff9700 1 -- 172.21.15.77:0/2000992959 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0027070 con 0x7f8cd81036b0 2024-05-27T08:31:38.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8cd4ff9700 1 -- 172.21.15.77:0/2000992959 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc0004120 con 0x7f8cd81036b0 2024-05-27T08:31:38.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8cd4ff9700 1 -- 172.21.15.77:0/2000992959 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000f460 con 0x7f8cd81036b0 2024-05-27T08:31:38.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd81a16e0 con 0x7f8cd81036b0 2024-05-27T08:31:38.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.891+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd81a1c10 con 0x7f8cd81036b0 2024-05-27T08:31:38.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.892+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd806a230 con 0x7f8cd81036b0 2024-05-27T08:31:38.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.893+0000 7f8cd4ff9700 1 -- 172.21.15.77:0/2000992959 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8cc002a070 con 0x7f8cd81036b0 2024-05-27T08:31:38.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.894+0000 7f8cd4ff9700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8cc40b2900 0x7f8cc40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:31:38.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.894+0000 7f8cd6ffd700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8cc40b2900 0x7f8cc40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:31:38.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.894+0000 7f8cd4ff9700 1 -- 172.21.15.77:0/2000992959 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7f8cc00e27e0 con 0x7f8cd81036b0 2024-05-27T08:31:38.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.894+0000 7f8cd6ffd700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8cc40b2900 0x7f8cc40b4dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8103510 tx=0x7f8cc8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:31:38.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:38.898+0000 7f8cd4ff9700 1 -- 172.21.15.77:0/2000992959 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8cc0031070 con 0x7f8cd81036b0 2024-05-27T08:31:39.112 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:39.111+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8cd80fe890 con 0x7f8cc40b2900 2024-05-27T08:31:39.710 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:39 smithi077 ceph-mon[32007]: osdmap e41: 6 total, 6 up, 6 in 2024-05-27T08:31:39.711 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:39 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:31:39.711 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:39 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:31:39.711 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:39 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:39.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:39 smithi092 ceph-mon[36049]: osdmap e41: 6 total, 6 up, 6 in 2024-05-27T08:31:39.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:39 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:31:39.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:39 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:31:39.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:39 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:39.774 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:39 smithi179 ceph-mon[36481]: osdmap e41: 6 total, 6 up, 6 in 2024-05-27T08:31:39.774 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:39 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:31:39.774 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:39 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:31:39.774 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:39 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:40.658 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:40 smithi077 ceph-mon[32007]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:40.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:40 smithi077 ceph-mon[32007]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:31:40.659 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:40 smithi077 ceph-mon[32007]: osdmap e42: 6 total, 6 up, 6 in 2024-05-27T08:31:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:40 smithi092 ceph-mon[36049]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:40 smithi092 ceph-mon[36049]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:31:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:40 smithi092 ceph-mon[36049]: osdmap e42: 6 total, 6 up, 6 in 2024-05-27T08:31:40.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:40 smithi179 ceph-mon[36481]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:40.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:40 smithi179 ceph-mon[36481]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:31:40.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:40 smithi179 ceph-mon[36481]: osdmap e42: 6 total, 6 up, 6 in 2024-05-27T08:31:42.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:42 smithi077 ceph-mon[32007]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:42.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:42 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2602957706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5"}]: dispatch 2024-05-27T08:31:42.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:42 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2602957706' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5"}]': finished 2024-05-27T08:31:42.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:42 smithi077 ceph-mon[32007]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T08:31:42.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:42 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:31:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:42 smithi092 ceph-mon[36049]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:42 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2602957706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5"}]: dispatch 2024-05-27T08:31:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:42 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2602957706' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5"}]': finished 2024-05-27T08:31:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:42 smithi092 ceph-mon[36049]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T08:31:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:42 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:31:42.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:42 smithi179 ceph-mon[36481]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:42.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:42 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2602957706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5"}]: dispatch 2024-05-27T08:31:42.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:42 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2602957706' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5"}]': finished 2024-05-27T08:31:42.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:42 smithi179 ceph-mon[36481]: osdmap e43: 7 total, 6 up, 7 in 2024-05-27T08:31:42.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:42 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:31:43.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:43 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/4144135014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:31:43.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:43 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/4144135014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:31:43.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:43 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/4144135014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:31:44.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:44 smithi092 ceph-mon[36049]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-27T08:31:44.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:44 smithi179 ceph-mon[36481]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-27T08:31:44.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:44 smithi077 ceph-mon[32007]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-05-27T08:31:46.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:46 smithi092 ceph-mon[36049]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-27T08:31:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:46 smithi179 ceph-mon[36481]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-27T08:31:46.867 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:46 smithi077 ceph-mon[32007]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-27T08:31:48.497 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:48 smithi077 ceph-mon[32007]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:31:48.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:48 smithi092 ceph-mon[36049]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:31:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:48 smithi179 ceph-mon[36481]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:31:50.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:50 smithi077 ceph-mon[32007]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:31:50.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T08:31:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:50 smithi077 ceph-mon[32007]: Deploying daemon osd.6 on smithi077 2024-05-27T08:31:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:50 smithi092 ceph-mon[36049]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:31:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T08:31:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:50 smithi092 ceph-mon[36049]: Deploying daemon osd.6 on smithi077 2024-05-27T08:31:50.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:50 smithi179 ceph-mon[36481]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:31:50.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-27T08:31:50.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:50.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:50 smithi179 ceph-mon[36481]: Deploying daemon osd.6 on smithi077 2024-05-27T08:31:52.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:52 smithi092 ceph-mon[36049]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:31:52.791 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:52 smithi077 ceph-mon[32007]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:31:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:52 smithi179 ceph-mon[36481]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:31:54.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:54 smithi092 ceph-mon[36049]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:54.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:54 smithi179 ceph-mon[36481]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:54.865 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:54 smithi077 ceph-mon[32007]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:56 smithi092 ceph-mon[36049]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:56 smithi179 ceph-mon[36481]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:56.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:56 smithi077 ceph-mon[32007]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:56.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:56.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:56.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:31:58.540 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:58 smithi077 ceph-mon[32007]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:58.541 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:58.541 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:58.541 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:58.541 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:58.541 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:31:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:58.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:58 smithi092 ceph-mon[36049]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:58.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:58.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:58.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:58.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:58.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:31:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:58 smithi179 ceph-mon[36481]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:31:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:31:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:31:58.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:31:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:31:59.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.321+0000 7f8cd4ff9700 1 -- 172.21.15.77:0/2000992959 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8cd80fe890 con 0x7f8cc40b2900 2024-05-27T08:31:59.322 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 6 on host 'smithi077' 2024-05-27T08:31:59.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8cc40b2900 msgr2=0x7f8cc40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:59.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8cc40b2900 0x7f8cc40b4dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8103510 tx=0x7f8cc8008040 comp rx=0 tx=0).stop 2024-05-27T08:31:59.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 msgr2=0x7f8cd8199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:31:59.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 0x7f8cd8199d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0000c00 tx=0x7f8cc0005990 comp rx=0 tx=0).stop 2024-05-27T08:31:59.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 shutdown_connections 2024-05-27T08:31:59.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8cc40b2900 0x7f8cc40b4dc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:59.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cd8104050 0x7f8cd81a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:59.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd81024b0 0x7f8cd81997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:59.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 --2- 172.21.15.77:0/2000992959 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cd81036b0 0x7f8cd8199d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:31:59.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 >> 172.21.15.77:0/2000992959 conn(0x7f8cd80fda70 msgr2=0x7f8cd80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:31:59.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 shutdown_connections 2024-05-27T08:31:59.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:31:59.324+0000 7f8cdd41a700 1 -- 172.21.15.77:0/2000992959 wait complete. 2024-05-27T08:32:00.048 DEBUG:teuthology.orchestra.run.smithi077:osd.6> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.6.service 2024-05-27T08:32:00.051 INFO:tasks.cephadm:Deploying osd.7 on smithi092 with /dev/vg_nvme/lv_2... 2024-05-27T08:32:00.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T08:32:00.291 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:32:00.412 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:00 smithi092 ceph-mon[36049]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:00.412 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:00.412 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:00.412 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:00.412 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:00 smithi092 ceph-mon[36049]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T08:32:00.527 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:00 smithi077 ceph-mon[32007]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:00.527 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:00.527 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:00.527 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:00.527 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:00 smithi077 ceph-mon[32007]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T08:32:00.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:00 smithi179 ceph-mon[36481]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:00.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:00.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:00.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:00.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:00 smithi179 ceph-mon[36481]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-27T08:32:01.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:01 smithi077 ceph-mon[32007]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T08:32:01.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:01 smithi077 ceph-mon[32007]: osdmap e44: 7 total, 6 up, 7 in 2024-05-27T08:32:01.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:01 smithi077 ceph-mon[32007]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:32:01.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:01.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:01 smithi179 ceph-mon[36481]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T08:32:01.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:01 smithi179 ceph-mon[36481]: osdmap e44: 7 total, 6 up, 7 in 2024-05-27T08:32:01.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:01 smithi179 ceph-mon[36481]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:32:01.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:01.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:01 smithi092 ceph-mon[36049]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-27T08:32:01.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:01 smithi092 ceph-mon[36049]: osdmap e44: 7 total, 6 up, 7 in 2024-05-27T08:32:01.625 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:01 smithi092 ceph-mon[36049]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:32:01.626 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T08:32:02.315 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T08:32:02.315 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-27T08:32:02.316 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-27T08:32:02.316 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012563 s, 835 MB/s 2024-05-27T08:32:02.316 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-27T08:32:02.595 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:02 smithi077 ceph-mon[32007]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:02.595 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:02 smithi077 ceph-mon[32007]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:32:02.595 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:02 smithi077 ceph-mon[32007]: osdmap e45: 7 total, 6 up, 7 in 2024-05-27T08:32:02.595 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:02.596 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:02.596 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:02 smithi077 ceph-mon[32007]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' 2024-05-27T08:32:02.596 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:32:02 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:32:02.188+0000 7f2137fc7700 -1 osd.6 0 waiting for initial osdmap 2024-05-27T08:32:02.596 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:32:02 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:32:02.285+0000 7f212fdb2700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:32:02.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:02 smithi179 ceph-mon[36481]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:02.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:02 smithi179 ceph-mon[36481]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:32:02.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:02 smithi179 ceph-mon[36481]: osdmap e45: 7 total, 6 up, 7 in 2024-05-27T08:32:02.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:02.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:02.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:02 smithi179 ceph-mon[36481]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' 2024-05-27T08:32:02.621 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:02 smithi092 ceph-mon[36049]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:02.621 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:02 smithi092 ceph-mon[36049]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:32:02.621 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:02 smithi092 ceph-mon[36049]: osdmap e45: 7 total, 6 up, 7 in 2024-05-27T08:32:02.621 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:02.621 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:02.621 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:02 smithi092 ceph-mon[36049]: from='osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520]' entity='osd.6' 2024-05-27T08:32:02.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_2 2024-05-27T08:32:03.192 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:32:03.573 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:03 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:32:03.573 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:03 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:32:03.573 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:03 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:03.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:03 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:32:03.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:03 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:32:03.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:03 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:03.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:03 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:32:03.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:03 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:32:03.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:03 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:04.348 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:04.348 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] boot 2024-05-27T08:32:04.348 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: osdmap e46: 7 total, 7 up, 7 in 2024-05-27T08:32:04.348 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi077 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi077 to 5696M 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:04.349 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] boot 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: osdmap e46: 7 total, 7 up, 7 in 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi077 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi077 to 5696M 2024-05-27T08:32:04.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:04.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:04.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: osd.6 [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] boot 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: osdmap e46: 7 total, 7 up, 7 in 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi077 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:04.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi077 to 5696M 2024-05-27T08:32:04.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:04.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:04.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:04.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 -- 172.21.15.92:0/2218324506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d8104040 msgr2=0x7fa7d810a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 --2- 172.21.15.92:0/2218324506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d8104040 0x7fa7d810a8f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b0d0 tx=0x7fa7d000b3e0 comp rx=0 tx=0).stop 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 -- 172.21.15.92:0/2218324506 shutdown_connections 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 --2- 172.21.15.92:0/2218324506 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d8104040 0x7fa7d810a8f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 --2- 172.21.15.92:0/2218324506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7d81024a0 0x7fa7d81028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 --2- 172.21.15.92:0/2218324506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7d81036a0 0x7fa7d8103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 -- 172.21.15.92:0/2218324506 >> 172.21.15.92:0/2218324506 conn(0x7fa7d80fda40 msgr2=0x7fa7d80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 -- 172.21.15.92:0/2218324506 shutdown_connections 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.837+0000 7fa7de290700 1 -- 172.21.15.92:0/2218324506 wait complete. 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7de290700 1 Processor -- start 2024-05-27T08:32:04.840 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7de290700 1 -- start start 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7de290700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7d81024a0 0x7fa7d8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7de290700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d81036a0 0x7fa7d8199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7de290700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7d8104040 0x7fa7d81a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7de290700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d810dfc0 con 0x7fa7d8104040 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7de290700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d810de40 con 0x7fa7d81036a0 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7de290700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d810e140 con 0x7fa7d81024a0 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d81036a0 0x7fa7d8199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d77fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d81036a0 0x7fa7d8199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:36452/0 (socket says 172.21.15.92:36452) 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d77fe700 1 -- 172.21.15.92:0/1241368546 learned_addr learned my addr 172.21.15.92:0/1241368546 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d7fff700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7d81024a0 0x7fa7d8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7dc82d700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7d8104040 0x7fa7d81a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d77fe700 1 -- 172.21.15.92:0/1241368546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7d81024a0 msgr2=0x7fa7d8199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:04.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d77fe700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7d81024a0 0x7fa7d8199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d77fe700 1 -- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7d8104040 msgr2=0x7fa7d81a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d77fe700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7d8104040 0x7fa7d81a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d77fe700 1 -- 172.21.15.92:0/1241368546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d81a13d0 con 0x7fa7d81036a0 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7d7fff700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7d81024a0 0x7fa7d8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.838+0000 7fa7dc82d700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7d8104040 0x7fa7d81a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.839+0000 7fa7d77fe700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d81036a0 0x7fa7d8199cb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800d950 tx=0x7fa7c800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.839+0000 7fa7d57fa700 1 -- 172.21.15.92:0/1241368546 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8016070 con 0x7fa7d81036a0 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.839+0000 7fa7d57fa700 1 -- 172.21.15.92:0/1241368546 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c8010460 con 0x7fa7d81036a0 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.839+0000 7fa7d57fa700 1 -- 172.21.15.92:0/1241368546 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c800f600 con 0x7fa7d81036a0 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.839+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d81a16c0 con 0x7fa7d81036a0 2024-05-27T08:32:04.842 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.839+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d81a1cd0 con 0x7fa7d81036a0 2024-05-27T08:32:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.840+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d806a230 con 0x7fa7d81036a0 2024-05-27T08:32:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.842+0000 7fa7d57fa700 1 -- 172.21.15.92:0/1241368546 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa7c8019070 con 0x7fa7d81036a0 2024-05-27T08:32:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.842+0000 7fa7d57fa700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa7c40b2900 0x7fa7c40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.843+0000 7fa7d57fa700 1 -- 172.21.15.92:0/1241368546 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fa7c80e1070 con 0x7fa7d81036a0 2024-05-27T08:32:04.843 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.843+0000 7fa7d7fff700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa7c40b2900 0x7fa7c40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:04.844 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.843+0000 7fa7d7fff700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa7c40b2900 0x7fa7c40b4dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8103500 tx=0x7fa7c0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:32:04.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:04.846+0000 7fa7d57fa700 1 -- 172.21.15.92:0/1241368546 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa7c80a9aa0 con 0x7fa7d81036a0 2024-05-27T08:32:05.060 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:05.060+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa7d80ffc60 con 0x7fa7c40b2900 2024-05-27T08:32:05.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:05 smithi092 ceph-mon[36049]: osdmap e47: 7 total, 7 up, 7 in 2024-05-27T08:32:05.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:05.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:05.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:05.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:05 smithi179 ceph-mon[36481]: osdmap e47: 7 total, 7 up, 7 in 2024-05-27T08:32:05.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:05.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:05.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:05.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:05 smithi077 ceph-mon[32007]: osdmap e47: 7 total, 7 up, 7 in 2024-05-27T08:32:05.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:05.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:05.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:06.591 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:06 smithi092 ceph-mon[36049]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:06.591 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:06 smithi092 ceph-mon[36049]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:06.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:06 smithi179 ceph-mon[36481]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:06.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:06 smithi179 ceph-mon[36481]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:06.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:06 smithi077 ceph-mon[32007]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:06.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:06 smithi077 ceph-mon[32007]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:08.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:08 smithi092 ceph-mon[36049]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:08.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:08 smithi092 ceph-mon[36049]: from='client.? 172.21.15.92:0/536717020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]: dispatch 2024-05-27T08:32:08.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:08 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]: dispatch 2024-05-27T08:32:08.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:08 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]': finished 2024-05-27T08:32:08.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:08 smithi092 ceph-mon[36049]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T08:32:08.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:08.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:08 smithi179 ceph-mon[36481]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:08.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:08 smithi179 ceph-mon[36481]: from='client.? 172.21.15.92:0/536717020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]: dispatch 2024-05-27T08:32:08.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:08 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]: dispatch 2024-05-27T08:32:08.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:08 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]': finished 2024-05-27T08:32:08.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:08 smithi179 ceph-mon[36481]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T08:32:08.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:08.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:08 smithi077 ceph-mon[32007]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:08.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:08 smithi077 ceph-mon[32007]: from='client.? 172.21.15.92:0/536717020' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]: dispatch 2024-05-27T08:32:08.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:08 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]: dispatch 2024-05-27T08:32:08.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:08 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6262ef1-767f-4b5f-a376-f6a4cb631146"}]': finished 2024-05-27T08:32:08.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:08 smithi077 ceph-mon[32007]: osdmap e48: 8 total, 7 up, 8 in 2024-05-27T08:32:08.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:09.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:09 smithi092 ceph-mon[36049]: from='client.? 172.21.15.92:0/2504593227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:32:09.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:09 smithi179 ceph-mon[36481]: from='client.? 172.21.15.92:0/2504593227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:32:09.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:09 smithi077 ceph-mon[32007]: from='client.? 172.21.15.92:0/2504593227' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:32:10.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:10 smithi179 ceph-mon[36481]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:10.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:10 smithi077 ceph-mon[32007]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:10.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:10 smithi092 ceph-mon[36049]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:12.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:12 smithi179 ceph-mon[36481]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:12.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:12 smithi077 ceph-mon[32007]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:12.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:12 smithi092 ceph-mon[36049]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:14.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:14 smithi179 ceph-mon[36481]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:14.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:14 smithi077 ceph-mon[32007]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:14.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:14 smithi092 ceph-mon[36049]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:15.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:15 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T08:32:15.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:15 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:15.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:15 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T08:32:15.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:15 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:15.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:15 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-27T08:32:15.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:15 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:16.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:16 smithi179 ceph-mon[36481]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:16.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:16 smithi179 ceph-mon[36481]: Deploying daemon osd.7 on smithi092 2024-05-27T08:32:16.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:16 smithi077 ceph-mon[32007]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:16.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:16 smithi077 ceph-mon[32007]: Deploying daemon osd.7 on smithi092 2024-05-27T08:32:16.722 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:16 smithi092 ceph-mon[36049]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:16.722 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:16 smithi092 ceph-mon[36049]: Deploying daemon osd.7 on smithi092 2024-05-27T08:32:18.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:18 smithi179 ceph-mon[36481]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:18.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:18 smithi077 ceph-mon[32007]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:18.664 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:18 smithi092 ceph-mon[36049]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:20.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:20 smithi179 ceph-mon[36481]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:20.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:20 smithi077 ceph-mon[32007]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:20.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:20 smithi092 ceph-mon[36049]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:21.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:21 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:21.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:21 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:21.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:21 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:21 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:21 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:21 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:21.887 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:21 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:21.887 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:21 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:21.887 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:21 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:22 smithi092 ceph-mon[36049]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:22.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:22 smithi179 ceph-mon[36481]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:22.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:22 smithi077 ceph-mon[32007]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:23.874 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.872+0000 7fa7d57fa700 1 -- 172.21.15.92:0/1241368546 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa7d80ffc60 con 0x7fa7c40b2900 2024-05-27T08:32:23.874 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 7 on host 'smithi092' 2024-05-27T08:32:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa7c40b2900 msgr2=0x7fa7c40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa7c40b2900 0x7fa7c40b4dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8103500 tx=0x7fa7c0008040 comp rx=0 tx=0).stop 2024-05-27T08:32:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d81036a0 msgr2=0x7fa7d8199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:23.876 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d81036a0 0x7fa7d8199cb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800d950 tx=0x7fa7c800dd10 comp rx=0 tx=0).stop 2024-05-27T08:32:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 shutdown_connections 2024-05-27T08:32:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa7d8104040 0x7fa7d81a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fa7c40b2900 0x7fa7c40b4dc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7d81024a0 0x7fa7d8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 --2- 172.21.15.92:0/1241368546 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7d81036a0 0x7fa7d8199cb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 >> 172.21.15.92:0/1241368546 conn(0x7fa7d80fda40 msgr2=0x7fa7d80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:32:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 shutdown_connections 2024-05-27T08:32:23.877 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:32:23.875+0000 7fa7de290700 1 -- 172.21.15.92:0/1241368546 wait complete. 2024-05-27T08:32:24.427 DEBUG:teuthology.orchestra.run.smithi092:osd.7> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.7.service 2024-05-27T08:32:24.430 INFO:tasks.cephadm:Deploying osd.8 on smithi179 with /dev/vg_nvme/lv_2... 2024-05-27T08:32:24.430 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-27T08:32:24.675 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:32:24.736 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:32:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:32:24.477+0000 7fa396bc37c0 -1 osd.7 0 log_to_monitors true 2024-05-27T08:32:25.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:24 smithi179 ceph-mon[36481]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:25.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:24 smithi179 ceph-mon[36481]: from='osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T08:32:25.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:24 smithi179 ceph-mon[36481]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T08:32:25.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:24 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:24 smithi077 ceph-mon[32007]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:24 smithi077 ceph-mon[32007]: from='osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T08:32:25.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:24 smithi077 ceph-mon[32007]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T08:32:25.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:24 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.199 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:24 smithi092 ceph-mon[36049]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:25.199 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.199 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.200 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:24 smithi092 ceph-mon[36049]: from='osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T08:32:25.200 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:24 smithi092 ceph-mon[36049]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-27T08:32:25.200 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:25.200 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:24 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:26.065 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:25 smithi179 ceph-mon[36481]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T08:32:26.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:25 smithi179 ceph-mon[36481]: osdmap e49: 8 total, 7 up, 8 in 2024-05-27T08:32:26.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:25 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:26.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:25 smithi179 ceph-mon[36481]: from='osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:32:26.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:25 smithi179 ceph-mon[36481]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:32:26.114 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:25 smithi092 ceph-mon[36049]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T08:32:26.114 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:25 smithi092 ceph-mon[36049]: osdmap e49: 8 total, 7 up, 8 in 2024-05-27T08:32:26.115 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:25 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:26.115 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:25 smithi092 ceph-mon[36049]: from='osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:32:26.115 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:25 smithi092 ceph-mon[36049]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:32:26.116 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:32:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:32:25.863+0000 7fa38c227700 -1 osd.7 0 waiting for initial osdmap 2024-05-27T08:32:26.116 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:32:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:32:25.873+0000 7fa38801a700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:32:26.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:25 smithi077 ceph-mon[32007]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-27T08:32:26.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:25 smithi077 ceph-mon[32007]: osdmap e49: 8 total, 7 up, 8 in 2024-05-27T08:32:26.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:25 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:26.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:25 smithi077 ceph-mon[32007]: from='osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:32:26.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:25 smithi077 ceph-mon[32007]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:32:26.643 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-27T08:32:26.643 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-27T08:32:26.644 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-05-27T08:32:26.644 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-05-27T08:32:26.644 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113548 s, 923 MB/s 2024-05-27T08:32:26.644 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-05-27T08:32:26.999 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:26 smithi179 ceph-mon[36481]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:27.000 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:26 smithi179 ceph-mon[36481]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:32:27.000 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:26 smithi179 ceph-mon[36481]: osdmap e50: 8 total, 7 up, 8 in 2024-05-27T08:32:27.000 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:26 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:27.000 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:26 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:27.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:26 smithi077 ceph-mon[32007]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:27.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:26 smithi077 ceph-mon[32007]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:32:27.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:26 smithi077 ceph-mon[32007]: osdmap e50: 8 total, 7 up, 8 in 2024-05-27T08:32:27.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:26 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:27.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:26 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:27.152 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:26 smithi092 ceph-mon[36049]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-05-27T08:32:27.152 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:26 smithi092 ceph-mon[36049]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:32:27.152 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:26 smithi092 ceph-mon[36049]: osdmap e50: 8 total, 7 up, 8 in 2024-05-27T08:32:27.152 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:26 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:27.153 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:26 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:27.321 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-05-27T08:32:27.557 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:32:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:27 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:32:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:27 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:32:28.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:27 smithi179 ceph-mon[36481]: osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] boot 2024-05-27T08:32:28.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:27 smithi179 ceph-mon[36481]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T08:32:28.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:28.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:27 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:32:28.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:27 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:32:28.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:27 smithi077 ceph-mon[32007]: osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] boot 2024-05-27T08:32:28.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:27 smithi077 ceph-mon[32007]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T08:32:28.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:28.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:27 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:32:28.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:27 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:32:28.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:27 smithi092 ceph-mon[36049]: osd.7 [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] boot 2024-05-27T08:32:28.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:27 smithi092 ceph-mon[36049]: osdmap e51: 8 total, 8 up, 8 in 2024-05-27T08:32:28.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:32:29.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 -- 172.21.15.179:0/2331153976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 msgr2=0x7f39c81028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:29.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 --2- 172.21.15.179:0/2331153976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 0x7f39c81028e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009990 tx=0x7f39c4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:32:29.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 -- 172.21.15.179:0/2331153976 shutdown_connections 2024-05-27T08:32:29.070 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 --2- 172.21.15.179:0/2331153976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c8104060 0x7f39c810a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 --2- 172.21.15.179:0/2331153976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 0x7f39c81028e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 --2- 172.21.15.179:0/2331153976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39c81036c0 0x7f39c8103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 -- 172.21.15.179:0/2331153976 >> 172.21.15.179:0/2331153976 conn(0x7f39c80fdaa0 msgr2=0x7f39c80ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 -- 172.21.15.179:0/2331153976 shutdown_connections 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 -- 172.21.15.179:0/2331153976 wait complete. 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.066+0000 7f39cef30700 1 Processor -- start 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cef30700 1 -- start start 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cef30700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 0x7f39c806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cef30700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c81036c0 0x7f39c81a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cef30700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39c8104060 0x7f39c81a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:29.071 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cef30700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810df50 con 0x7f39c81036c0 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cef30700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810ddd0 con 0x7f39c8104060 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cef30700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39c810e0d0 con 0x7f39c81024c0 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 0x7f39c806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 0x7f39c806c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:33598/0 (socket says 172.21.15.179:33598) 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 -- 172.21.15.179:0/341625980 learned_addr learned my addr 172.21.15.179:0/341625980 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cd4cd700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39c8104060 0x7f39c81a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39bffff700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c81036c0 0x7f39c81a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 -- 172.21.15.179:0/341625980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39c8104060 msgr2=0x7f39c81a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:29.072 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39c8104060 0x7f39c81a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 -- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c81036c0 msgr2=0x7f39c81a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c81036c0 0x7f39c81a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 -- 172.21.15.179:0/341625980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c81a3520 con 0x7f39c81024c0 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39ccccc700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 0x7f39c806c7f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39c4000c00 tx=0x7f39c4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39bdffb700 1 -- 172.21.15.179:0/341625980 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4026070 con 0x7f39c81024c0 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c81a37b0 con 0x7f39c81024c0 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.067+0000 7f39bdffb700 1 -- 172.21.15.179:0/341625980 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c40040c0 con 0x7f39c81024c0 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.068+0000 7f39bdffb700 1 -- 172.21.15.179:0/341625980 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4005290 con 0x7f39c81024c0 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.068+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81a3c60 con 0x7f39c81024c0 2024-05-27T08:32:29.073 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.069+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c806a230 con 0x7f39c81024c0 2024-05-27T08:32:29.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.070+0000 7f39bdffb700 1 -- 172.21.15.179:0/341625980 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f39c4029070 con 0x7f39c81024c0 2024-05-27T08:32:29.074 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.071+0000 7f39bdffb700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39b00b2880 0x7f39b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:29.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.071+0000 7f39bffff700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39b00b2880 0x7f39b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:29.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.071+0000 7f39bdffb700 1 -- 172.21.15.179:0/341625980 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5397+0+0 (secure 0 0 0) 0x7f39c40e2570 con 0x7f39c81024c0 2024-05-27T08:32:29.075 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.071+0000 7f39bffff700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39b00b2880 0x7f39b00b4d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39c806cd70 tx=0x7f39b800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:32:29.076 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.075+0000 7f39bdffb700 1 -- 172.21.15.179:0/341625980 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39c4030020 con 0x7f39c81024c0 2024-05-27T08:32:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi092 2024-05-27T08:32:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: osdmap e52: 8 total, 8 up, 8 in 2024-05-27T08:32:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:29.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:28 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi092 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: osdmap e52: 8 total, 8 up, 8 in 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-27T08:32:29.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:29.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:29.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:29.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:29.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:29.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:28 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:29.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi092 2024-05-27T08:32:29.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: osdmap e52: 8 total, 8 up, 8 in 2024-05-27T08:32:29.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi092 to 5696M 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:29.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:29.238 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:29.238 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:28 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:29.289 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:29.288+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f39c8063980 con 0x7f39b00b2880 2024-05-27T08:32:30.216 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:29 smithi179 ceph-mon[36481]: osdmap e53: 8 total, 8 up, 8 in 2024-05-27T08:32:30.216 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:29 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:30.216 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:29 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:30.216 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:29 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:30.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:29 smithi092 ceph-mon[36049]: osdmap e53: 8 total, 8 up, 8 in 2024-05-27T08:32:30.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:29 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:30.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:29 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:30.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:29 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:30.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:29 smithi077 ceph-mon[32007]: osdmap e53: 8 total, 8 up, 8 in 2024-05-27T08:32:30.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:29 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:31.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:30 smithi077 ceph-mon[32007]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:31.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:30 smithi077 ceph-mon[32007]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:31.157 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:30 smithi179 ceph-mon[36481]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:31.157 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:30 smithi179 ceph-mon[36481]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:31.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:30 smithi092 ceph-mon[36049]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:31.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:30 smithi092 ceph-mon[36049]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:31 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/2383418998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]: dispatch 2024-05-27T08:32:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:31 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]: dispatch 2024-05-27T08:32:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:31 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]': finished 2024-05-27T08:32:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:31 smithi092 ceph-mon[36049]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T08:32:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:32.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:31 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/2383418998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]: dispatch 2024-05-27T08:32:32.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:31 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]: dispatch 2024-05-27T08:32:32.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:31 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]': finished 2024-05-27T08:32:32.264 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:31 smithi179 ceph-mon[36481]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T08:32:32.265 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:32.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:31 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/2383418998' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]: dispatch 2024-05-27T08:32:32.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:31 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]: dispatch 2024-05-27T08:32:32.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:31 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "018d4143-7dfd-40d3-9b26-758ce6acb695"}]': finished 2024-05-27T08:32:32.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:31 smithi077 ceph-mon[32007]: osdmap e54: 9 total, 8 up, 9 in 2024-05-27T08:32:32.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:32 smithi092 ceph-mon[36049]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:32 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/969434390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:32:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:32 smithi179 ceph-mon[36481]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:32 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/969434390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:32:33.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:32 smithi077 ceph-mon[32007]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:33.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:32 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/969434390' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:32:35.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:34 smithi092 ceph-mon[36049]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T08:32:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:34 smithi179 ceph-mon[36481]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T08:32:35.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:34 smithi077 ceph-mon[32007]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-27T08:32:37.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:36 smithi092 ceph-mon[36049]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-27T08:32:37.267 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:36 smithi179 ceph-mon[36481]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-27T08:32:37.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:36 smithi077 ceph-mon[32007]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-27T08:32:39.197 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:38 smithi179 ceph-mon[36481]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-27T08:32:39.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:38 smithi092 ceph-mon[36049]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-27T08:32:39.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:38 smithi077 ceph-mon[32007]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-27T08:32:40.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:39 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T08:32:40.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:39 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:40.234 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:39 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T08:32:40.234 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:39 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:40.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:39 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-27T08:32:40.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:39 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:41.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:40 smithi092 ceph-mon[36049]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:32:41.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:40 smithi092 ceph-mon[36049]: Deploying daemon osd.8 on smithi179 2024-05-27T08:32:41.273 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:40 smithi179 ceph-mon[36481]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:32:41.273 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:40 smithi179 ceph-mon[36481]: Deploying daemon osd.8 on smithi179 2024-05-27T08:32:41.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:40 smithi077 ceph-mon[32007]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:32:41.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:40 smithi077 ceph-mon[32007]: Deploying daemon osd.8 on smithi179 2024-05-27T08:32:43.225 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:42 smithi179 ceph-mon[36481]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:32:43.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:42 smithi092 ceph-mon[36049]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:32:43.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:42 smithi077 ceph-mon[32007]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:32:45.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:44 smithi092 ceph-mon[36049]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:45.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:45.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:45.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:45.339 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:44 smithi179 ceph-mon[36481]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:45.339 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:45.339 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:45.339 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:45.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:44 smithi077 ceph-mon[32007]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:45.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:45.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:45.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:47.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:46 smithi092 ceph-mon[36049]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:47.290 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:46 smithi179 ceph-mon[36481]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:47.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:46 smithi077 ceph-mon[32007]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:48.560 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.558+0000 7f39bdffb700 1 -- 172.21.15.179:0/341625980 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f39c8063980 con 0x7f39b00b2880 2024-05-27T08:32:48.560 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 8 on host 'smithi179' 2024-05-27T08:32:48.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39b00b2880 msgr2=0x7f39b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:48.562 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39b00b2880 0x7f39b00b4d40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39c806cd70 tx=0x7f39b800a380 comp rx=0 tx=0).stop 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 msgr2=0x7f39c806c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 0x7f39c806c7f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39c4000c00 tx=0x7f39c4009580 comp rx=0 tx=0).stop 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 shutdown_connections 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39c8104060 0x7f39c81a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39b00b2880 0x7f39b00b4d40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c81024c0 0x7f39c806c7f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 --2- 172.21.15.179:0/341625980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39c81036c0 0x7f39c81a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 >> 172.21.15.179:0/341625980 conn(0x7f39c80fdaa0 msgr2=0x7f39c8106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 shutdown_connections 2024-05-27T08:32:48.563 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:32:48.561+0000 7f39cef30700 1 -- 172.21.15.179:0/341625980 wait complete. 2024-05-27T08:32:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:48 smithi179 ceph-mon[36481]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:48.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:48.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:48 smithi077 ceph-mon[32007]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:48.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:48 smithi092 ceph-mon[36049]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:49.151 DEBUG:teuthology.orchestra.run.smithi179:osd.8> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.8.service 2024-05-27T08:32:49.153 INFO:tasks.cephadm:Deploying osd.9 on smithi077 with /dev/vg_nvme/lv_1... 2024-05-27T08:32:49.153 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T08:32:49.406 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:32:49.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:49 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:49.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:49 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:49.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:49 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:49.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:49 smithi077 ceph-mon[32007]: from='osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T08:32:49.808 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:49 smithi077 ceph-mon[32007]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T08:32:49.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:49 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:49.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:49 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:49.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:49 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:49.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:49 smithi179 ceph-mon[36481]: from='osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T08:32:49.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:49 smithi179 ceph-mon[36481]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T08:32:49.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:49 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:49.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:49 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:49.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:49 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:32:49.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:49 smithi092 ceph-mon[36049]: from='osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T08:32:49.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:49 smithi092 ceph-mon[36049]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-27T08:32:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:50 smithi092 ceph-mon[36049]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:50 smithi092 ceph-mon[36049]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T08:32:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:50 smithi092 ceph-mon[36049]: osdmap e55: 9 total, 8 up, 9 in 2024-05-27T08:32:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:50 smithi092 ceph-mon[36049]: from='osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:32:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:50 smithi092 ceph-mon[36049]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:32:50.811 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:50 smithi179 ceph-mon[36481]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:50.811 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:50 smithi179 ceph-mon[36481]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T08:32:50.811 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:50 smithi179 ceph-mon[36481]: osdmap e55: 9 total, 8 up, 9 in 2024-05-27T08:32:50.811 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:50.811 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:50 smithi179 ceph-mon[36481]: from='osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:32:50.811 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:50 smithi179 ceph-mon[36481]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:32:50.812 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:32:50 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:32:50.556+0000 7f45c6317700 -1 osd.8 0 waiting for initial osdmap 2024-05-27T08:32:50.812 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:32:50 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:32:50.566+0000 7f45c210a700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:32:50.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:50 smithi077 ceph-mon[32007]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:50.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:50 smithi077 ceph-mon[32007]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-27T08:32:50.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:50 smithi077 ceph-mon[32007]: osdmap e55: 9 total, 8 up, 9 in 2024-05-27T08:32:50.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:50.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:50 smithi077 ceph-mon[32007]: from='osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:32:50.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:50 smithi077 ceph-mon[32007]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:32:51.624 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T08:32:51.625 INFO:teuthology.orchestra.run.smithi077.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T08:32:51.625 INFO:teuthology.orchestra.run.smithi077.stderr: stderr: 10+0 records in 2024-05-27T08:32:51.625 INFO:teuthology.orchestra.run.smithi077.stderr:10+0 records out 2024-05-27T08:32:51.625 INFO:teuthology.orchestra.run.smithi077.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012154 s, 863 MB/s 2024-05-27T08:32:51.625 INFO:teuthology.orchestra.run.smithi077.stderr:--> Zapping successful for: 2024-05-27T08:32:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:51 smithi179 ceph-mon[36481]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:32:51.697 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:51 smithi179 ceph-mon[36481]: osdmap e56: 9 total, 8 up, 9 in 2024-05-27T08:32:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:51.698 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:51.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:51 smithi077 ceph-mon[32007]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:32:51.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:51 smithi077 ceph-mon[32007]: osdmap e56: 9 total, 8 up, 9 in 2024-05-27T08:32:51.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:51.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:51.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:51 smithi092 ceph-mon[36049]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:32:51.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:51 smithi092 ceph-mon[36049]: osdmap e56: 9 total, 8 up, 9 in 2024-05-27T08:32:51.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:51.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:52.376 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2024-05-27T08:32:52.613 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:32:52.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:32:52.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:32:52.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:52.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:52.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] boot 2024-05-27T08:32:52.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T08:32:52.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:52.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi179 2024-05-27T08:32:52.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.632 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.632 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.632 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.632 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.632 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi179 to 5696M 2024-05-27T08:32:52.632 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.633 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:52.633 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:52.633 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:32:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:32:52.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] boot 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi179 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi179 to 5696M 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:52.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:52.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:32:52.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:32:52.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-05-27T08:32:52.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:52.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: osd.8 [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] boot 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: osdmap e57: 9 total, 9 up, 9 in 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi179 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi179 to 5696M 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:32:52.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:32:53.666 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:53 smithi077 ceph-mon[32007]: osdmap e58: 9 total, 9 up, 9 in 2024-05-27T08:32:53.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:53 smithi179 ceph-mon[36481]: osdmap e58: 9 total, 9 up, 9 in 2024-05-27T08:32:53.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:53 smithi092 ceph-mon[36049]: osdmap e58: 9 total, 9 up, 9 in 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/3949126019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f181018b0 msgr2=0x7f0f18101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/3949126019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f181018b0 0x7f0f18101cd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009990 tx=0x7f0f08009ca0 comp rx=0 tx=0).stop 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/3949126019 shutdown_connections 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/3949126019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f18105230 0x7f0f1810bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/3949126019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f181018b0 0x7f0f18101cd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/3949126019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f18104890 0x7f0f18104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/3949126019 >> 172.21.15.77:0/3949126019 conn(0x7f0f180fd210 msgr2=0x7f0f180ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/3949126019 shutdown_connections 2024-05-27T08:32:54.177 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/3949126019 wait complete. 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 Processor -- start 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.174+0000 7f0f1e5e5700 1 -- start start 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f1e5e5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f181018b0 0x7f0f18199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f1e5e5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f18104890 0x7f0f18199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f1e5e5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f18105230 0x7f0f181a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f1e5e5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810f0d0 con 0x7f0f18104890 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f1e5e5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810ef50 con 0x7f0f181018b0 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f1e5e5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0f1810f250 con 0x7f0f18105230 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f18104890 0x7f0f18199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f18104890 0x7f0f18199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57072/0 (socket says 172.21.15.77:57072) 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 -- 172.21.15.77:0/1300896277 learned_addr learned my addr 172.21.15.77:0/1300896277 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f1cb82700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f18105230 0x7f0f181a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:54.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f17fff700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f181018b0 0x7f0f18199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 -- 172.21.15.77:0/1300896277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f18105230 msgr2=0x7f0f181a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f18105230 0x7f0f181a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 -- 172.21.15.77:0/1300896277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f181018b0 msgr2=0x7f0f18199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f181018b0 0x7f0f18199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 -- 172.21.15.77:0/1300896277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f181a13d0 con 0x7f0f18104890 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f17fff700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f181018b0 0x7f0f18199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.175+0000 7f0f177fe700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f18104890 0x7f0f18199cb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00b560 tx=0x7f0f0c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.176+0000 7f0f157fa700 1 -- 172.21.15.77:0/1300896277 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c015020 con 0x7f0f18104890 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.176+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f181a16c0 con 0x7f0f18104890 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.176+0000 7f0f157fa700 1 -- 172.21.15.77:0/1300896277 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f0c011ce0 con 0x7f0f18104890 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.176+0000 7f0f157fa700 1 -- 172.21.15.77:0/1300896277 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c010d20 con 0x7f0f18104890 2024-05-27T08:32:54.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.176+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f181a1c50 con 0x7f0f18104890 2024-05-27T08:32:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.177+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f1806a230 con 0x7f0f18104890 2024-05-27T08:32:54.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.178+0000 7f0f157fa700 1 -- 172.21.15.77:0/1300896277 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0f0c018020 con 0x7f0f18104890 2024-05-27T08:32:54.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.179+0000 7f0f157fa700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0f000b2900 0x7f0f000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:32:54.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.179+0000 7f0f17fff700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0f000b2900 0x7f0f000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:32:54.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.179+0000 7f0f157fa700 1 -- 172.21.15.77:0/1300896277 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5829+0+0 (secure 0 0 0) 0x7f0f0c0e14b0 con 0x7f0f18104890 2024-05-27T08:32:54.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.179+0000 7f0f17fff700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0f000b2900 0x7f0f000b4dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0f080038a0 tx=0x7f0f08005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:32:54.183 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.182+0000 7f0f157fa700 1 -- 172.21.15.77:0/1300896277 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f0c01f020 con 0x7f0f18104890 2024-05-27T08:32:54.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:32:54.398+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0f18063980 con 0x7f0f000b2900 2024-05-27T08:32:54.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:54 smithi179 ceph-mon[36481]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:32:54.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:54 smithi179 ceph-mon[36481]: osdmap e59: 9 total, 9 up, 9 in 2024-05-27T08:32:54.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:54.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:54.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:54.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:54 smithi077 ceph-mon[32007]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:32:54.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:54 smithi077 ceph-mon[32007]: osdmap e59: 9 total, 9 up, 9 in 2024-05-27T08:32:54.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:54.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:54.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:54.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:54 smithi092 ceph-mon[36049]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:32:54.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:54 smithi092 ceph-mon[36049]: osdmap e59: 9 total, 9 up, 9 in 2024-05-27T08:32:54.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:32:54.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:32:54.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:32:55.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:55 smithi179 ceph-mon[36481]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:55.876 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:55 smithi077 ceph-mon[32007]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:55.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:55 smithi092 ceph-mon[36049]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:32:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:56 smithi179 ceph-mon[36481]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:32:56.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:56 smithi077 ceph-mon[32007]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:32:56.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:56 smithi092 ceph-mon[36049]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:32:57.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:57 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1222306087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "058309ee-c447-48af-aec5-768f8c9f5222"}]: dispatch 2024-05-27T08:32:57.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:57 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1222306087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "058309ee-c447-48af-aec5-768f8c9f5222"}]': finished 2024-05-27T08:32:57.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:57 smithi179 ceph-mon[36481]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T08:32:57.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:32:57.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:57 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1222306087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "058309ee-c447-48af-aec5-768f8c9f5222"}]: dispatch 2024-05-27T08:32:57.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:57 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1222306087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "058309ee-c447-48af-aec5-768f8c9f5222"}]': finished 2024-05-27T08:32:57.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:57 smithi077 ceph-mon[32007]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T08:32:57.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:32:57.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:57 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1222306087' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "058309ee-c447-48af-aec5-768f8c9f5222"}]: dispatch 2024-05-27T08:32:57.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:57 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1222306087' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "058309ee-c447-48af-aec5-768f8c9f5222"}]': finished 2024-05-27T08:32:57.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:57 smithi092 ceph-mon[36049]: osdmap e60: 10 total, 9 up, 10 in 2024-05-27T08:32:57.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:32:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:58 smithi179 ceph-mon[36481]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-27T08:32:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:32:58 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1778551947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:32:58.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:58 smithi077 ceph-mon[32007]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-27T08:32:58.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:32:58 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1778551947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:32:58.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:58 smithi092 ceph-mon[36049]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-27T08:32:58.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:32:58 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1778551947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:33:00.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:00 smithi179 ceph-mon[36481]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-27T08:33:00.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:00 smithi077 ceph-mon[32007]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-27T08:33:00.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:00 smithi092 ceph-mon[36049]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-27T08:33:02.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:02 smithi077 ceph-mon[32007]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:33:02.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:02 smithi179 ceph-mon[36481]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:33:02.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:02 smithi092 ceph-mon[36049]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:33:04.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:04 smithi179 ceph-mon[36481]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:33:04.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:04 smithi077 ceph-mon[32007]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:33:04.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:04 smithi092 ceph-mon[36049]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:33:05.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T08:33:05.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:05.779 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:05 smithi077 ceph-mon[32007]: Deploying daemon osd.9 on smithi077 2024-05-27T08:33:05.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T08:33:05.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:05.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:05 smithi179 ceph-mon[36481]: Deploying daemon osd.9 on smithi077 2024-05-27T08:33:05.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-27T08:33:05.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:05.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:05 smithi092 ceph-mon[36049]: Deploying daemon osd.9 on smithi077 2024-05-27T08:33:06.825 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:06 smithi077 ceph-mon[32007]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:33:06.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:06 smithi179 ceph-mon[36481]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:33:06.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:06 smithi092 ceph-mon[36049]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-27T08:33:08.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:08 smithi179 ceph-mon[36481]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:08.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:08 smithi077 ceph-mon[32007]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:08.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:08 smithi092 ceph-mon[36049]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:10.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:10 smithi179 ceph-mon[36481]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:10.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:10 smithi077 ceph-mon[32007]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:10.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:10 smithi092 ceph-mon[36049]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:11.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:11 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:11.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:11.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:11.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:11 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:11 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:12.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:12 smithi092 ceph-mon[36049]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:13.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:12 smithi179 ceph-mon[36481]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:13.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:12 smithi077 ceph-mon[32007]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:14.623 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 9 on host 'smithi077' 2024-05-27T08:33:14.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.622+0000 7f0f157fa700 1 -- 172.21.15.77:0/1300896277 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0f18063980 con 0x7f0f000b2900 2024-05-27T08:33:14.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0f000b2900 msgr2=0x7f0f000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0f000b2900 0x7f0f000b4dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0f080038a0 tx=0x7f0f08005fd0 comp rx=0 tx=0).stop 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f18104890 msgr2=0x7f0f18199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f18104890 0x7f0f18199cb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00b560 tx=0x7f0f0c00b870 comp rx=0 tx=0).stop 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 shutdown_connections 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0f000b2900 0x7f0f000b4dc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f18105230 0x7f0f181a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0f181018b0 0x7f0f18199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 --2- 172.21.15.77:0/1300896277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f18104890 0x7f0f18199cb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.624+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 >> 172.21.15.77:0/1300896277 conn(0x7f0f180fd210 msgr2=0x7f0f18107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.625+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 shutdown_connections 2024-05-27T08:33:14.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:33:14.625+0000 7f0f1e5e5700 1 -- 172.21.15.77:0/1300896277 wait complete. 2024-05-27T08:33:14.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:14.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:14 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:14 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-27T08:33:15.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:15.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:14 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:15.308 DEBUG:teuthology.orchestra.run.smithi077:osd.9> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.9.service 2024-05-27T08:33:15.311 INFO:tasks.cephadm:Deploying osd.10 on smithi092 with /dev/vg_nvme/lv_1... 2024-05-27T08:33:15.311 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T08:33:15.561 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:33:15.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:15 smithi077 ceph-mon[32007]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T08:33:15.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:15 smithi077 ceph-mon[32007]: osdmap e61: 10 total, 9 up, 10 in 2024-05-27T08:33:15.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:15 smithi077 ceph-mon[32007]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:33:15.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:15 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:15.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:15 smithi092 ceph-mon[36049]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T08:33:15.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:15 smithi092 ceph-mon[36049]: osdmap e61: 10 total, 9 up, 10 in 2024-05-27T08:33:15.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:15 smithi092 ceph-mon[36049]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:33:15.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:15 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:16.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:15 smithi179 ceph-mon[36481]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-27T08:33:16.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:15 smithi179 ceph-mon[36481]: osdmap e61: 10 total, 9 up, 10 in 2024-05-27T08:33:16.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:15 smithi179 ceph-mon[36481]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-05-27T08:33:16.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:15 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:16.749 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:33:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:33:16.715+0000 7f7629c3b700 -1 osd.9 0 waiting for initial osdmap 2024-05-27T08:33:16.923 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:16 smithi092 ceph-mon[36049]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:16.924 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:16 smithi092 ceph-mon[36049]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:33:16.924 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:16 smithi092 ceph-mon[36049]: osdmap e62: 10 total, 9 up, 10 in 2024-05-27T08:33:16.924 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:16.924 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:16.924 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:16 smithi092 ceph-mon[36049]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' 2024-05-27T08:33:17.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:16 smithi179 ceph-mon[36481]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:17.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:16 smithi179 ceph-mon[36481]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:33:17.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:16 smithi179 ceph-mon[36481]: osdmap e62: 10 total, 9 up, 10 in 2024-05-27T08:33:17.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:17.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:17.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:16 smithi179 ceph-mon[36481]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' 2024-05-27T08:33:17.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:16 smithi077 ceph-mon[32007]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:17.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:16 smithi077 ceph-mon[32007]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-05-27T08:33:17.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:16 smithi077 ceph-mon[32007]: osdmap e62: 10 total, 9 up, 10 in 2024-05-27T08:33:17.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:17.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:17.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:16 smithi077 ceph-mon[32007]: from='osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537]' entity='osd.9' 2024-05-27T08:33:17.131 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:33:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:33:16.937+0000 7f7621a26700 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:33:17.523 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T08:33:17.523 INFO:teuthology.orchestra.run.smithi092.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T08:33:17.523 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10+0 records in 2024-05-27T08:33:17.523 INFO:teuthology.orchestra.run.smithi092.stderr:10+0 records out 2024-05-27T08:33:17.523 INFO:teuthology.orchestra.run.smithi092.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137323 s, 764 MB/s 2024-05-27T08:33:17.523 INFO:teuthology.orchestra.run.smithi092.stderr:--> Zapping successful for: 2024-05-27T08:33:17.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:17 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:33:17.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:17 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:33:17.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:17 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:17 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:33:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:17 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:33:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:17 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:18.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:17 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:33:18.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:17 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:33:18.117 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:17 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:18.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi092:vg_nvme/lv_1 2024-05-27T08:33:18.386 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:33:19.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:19.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] boot 2024-05-27T08:33:19.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: osdmap e63: 10 total, 10 up, 10 in 2024-05-27T08:33:19.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:19.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:19.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:19.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:18 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] boot 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: osdmap e63: 10 total, 10 up, 10 in 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:19.132 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:19.132 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:18 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: osd.9 [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] boot 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: osdmap e63: 10 total, 10 up, 10 in 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:19.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:18 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:19.820 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:19 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi077 2024-05-27T08:33:19.821 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:19 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi077 to 4272M 2024-05-27T08:33:19.821 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:19 smithi092 ceph-mon[36049]: osdmap e64: 10 total, 10 up, 10 in 2024-05-27T08:33:20.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.003+0000 7f5973265700 1 -- 172.21.15.92:0/2435715394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 msgr2=0x7f596c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:20.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.003+0000 7f5973265700 1 --2- 172.21.15.92:0/2435715394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 0x7f596c1028c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59680099c0 tx=0x7f5968009cd0 comp rx=0 tx=0).stop 2024-05-27T08:33:20.005 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.003+0000 7f5973265700 1 -- 172.21.15.92:0/2435715394 shutdown_connections 2024-05-27T08:33:20.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.003+0000 7f5973265700 1 --2- 172.21.15.92:0/2435715394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f596c104040 0x7f596c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:20.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.003+0000 7f5973265700 1 --2- 172.21.15.92:0/2435715394 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 0x7f596c1028c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:20.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.003+0000 7f5973265700 1 --2- 172.21.15.92:0/2435715394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f596c1036a0 0x7f596c103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:20.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.003+0000 7f5973265700 1 -- 172.21.15.92:0/2435715394 >> 172.21.15.92:0/2435715394 conn(0x7f596c0fda40 msgr2=0x7f596c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:33:20.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 -- 172.21.15.92:0/2435715394 shutdown_connections 2024-05-27T08:33:20.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 -- 172.21.15.92:0/2435715394 wait complete. 2024-05-27T08:33:20.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 Processor -- start 2024-05-27T08:33:20.006 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 -- start start 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 0x7f596c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f596c1036a0 0x7f596c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f596c104040 0x7f596c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f596c10df50 con 0x7f596c104040 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f596c10ddd0 con 0x7f596c1024a0 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5973265700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f596c10e0d0 con 0x7f596c1036a0 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5971001700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 0x7f596c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5971001700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 0x7f596c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:40542/0 (socket says 172.21.15.92:40542) 2024-05-27T08:33:20.008 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.004+0000 7f5971001700 1 -- 172.21.15.92:0/3275214005 learned_addr learned my addr 172.21.15.92:0/3275214005 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5971802700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f596c104040 0x7f596c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5970800700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f596c1036a0 0x7f596c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5971001700 1 -- 172.21.15.92:0/3275214005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f596c1036a0 msgr2=0x7f596c199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5971001700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f596c1036a0 0x7f596c199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5971001700 1 -- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f596c104040 msgr2=0x7f596c1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5971001700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f596c104040 0x7f596c1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5971001700 1 -- 172.21.15.92:0/3275214005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c1a1430 con 0x7f596c1024a0 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5971001700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 0x7f596c199820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f596800fa80 tx=0x7f596800fab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f59627fc700 1 -- 172.21.15.92:0/3275214005 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968027070 con 0x7f596c1024a0 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f59627fc700 1 -- 172.21.15.92:0/3275214005 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5968022920 con 0x7f596c1024a0 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c1a16c0 con 0x7f596c1024a0 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f59627fc700 1 -- 172.21.15.92:0/3275214005 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968009e60 con 0x7f596c1024a0 2024-05-27T08:33:20.009 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.005+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1a1ba0 con 0x7f596c1024a0 2024-05-27T08:33:20.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.006+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c06a230 con 0x7f596c1024a0 2024-05-27T08:33:20.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.007+0000 7f59627fc700 1 -- 172.21.15.92:0/3275214005 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f596802a070 con 0x7f596c1024a0 2024-05-27T08:33:20.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.008+0000 7f59627fc700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f59580b2880 0x7f59580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:33:20.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.008+0000 7f59627fc700 1 -- 172.21.15.92:0/3275214005 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6261+0+0 (secure 0 0 0) 0x7f59680e2510 con 0x7f596c1024a0 2024-05-27T08:33:20.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.008+0000 7f5970800700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f59580b2880 0x7f59580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:33:20.011 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.009+0000 7f5970800700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f59580b2880 0x7f59580b4d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f595c009bf0 tx=0x7f595c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:33:20.014 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.013+0000 7f59627fc700 1 -- 172.21.15.92:0/3275214005 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5968013070 con 0x7f596c1024a0 2024-05-27T08:33:20.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:19 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi077 2024-05-27T08:33:20.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:19 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi077 to 4272M 2024-05-27T08:33:20.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:19 smithi179 ceph-mon[36481]: osdmap e64: 10 total, 10 up, 10 in 2024-05-27T08:33:20.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:19 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi077 2024-05-27T08:33:20.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:19 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi077 to 4272M 2024-05-27T08:33:20.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:19 smithi077 ceph-mon[32007]: osdmap e64: 10 total, 10 up, 10 in 2024-05-27T08:33:20.223 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:20.222+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f596c06b9a0 con 0x7f59580b2880 2024-05-27T08:33:20.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:20 smithi077 ceph-mon[32007]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:20.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:20 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:33:20.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:20 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:33:20.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:20 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:21.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:20 smithi092 ceph-mon[36049]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:21.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:20 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:33:21.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:20 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:33:21.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:20 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:20 smithi179 ceph-mon[36481]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:20 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:33:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:20 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:33:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:20 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:21.921 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:21 smithi092 ceph-mon[36049]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:33:22.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:21 smithi179 ceph-mon[36481]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:33:22.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:21 smithi077 ceph-mon[32007]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi092:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:33:23.094 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:22 smithi092 ceph-mon[36049]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:23.094 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:22 smithi092 ceph-mon[36049]: from='client.? 172.21.15.92:0/3077314850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]: dispatch 2024-05-27T08:33:23.094 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:22 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]: dispatch 2024-05-27T08:33:23.094 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:22 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]': finished 2024-05-27T08:33:23.095 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:22 smithi092 ceph-mon[36049]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T08:33:23.095 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:22 smithi179 ceph-mon[36481]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:22 smithi179 ceph-mon[36481]: from='client.? 172.21.15.92:0/3077314850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]: dispatch 2024-05-27T08:33:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:22 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]: dispatch 2024-05-27T08:33:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:22 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]': finished 2024-05-27T08:33:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:22 smithi179 ceph-mon[36481]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T08:33:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:23.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:22 smithi077 ceph-mon[32007]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:23.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:22 smithi077 ceph-mon[32007]: from='client.? 172.21.15.92:0/3077314850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]: dispatch 2024-05-27T08:33:23.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:22 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]: dispatch 2024-05-27T08:33:23.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:22 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a68bc35-c24b-4bf2-911f-2f711c0bd59a"}]': finished 2024-05-27T08:33:23.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:22 smithi077 ceph-mon[32007]: osdmap e65: 11 total, 10 up, 11 in 2024-05-27T08:33:23.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:24.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:23 smithi179 ceph-mon[36481]: from='client.? 172.21.15.92:0/2318163231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:33:24.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:23 smithi077 ceph-mon[32007]: from='client.? 172.21.15.92:0/2318163231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:33:24.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:23 smithi092 ceph-mon[36049]: from='client.? 172.21.15.92:0/2318163231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:33:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:24 smithi179 ceph-mon[36481]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:24 smithi077 ceph-mon[32007]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:25.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:24 smithi092 ceph-mon[36049]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:26.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:26 smithi092 ceph-mon[36049]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:27.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:26 smithi179 ceph-mon[36481]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:27.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:26 smithi077 ceph-mon[32007]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:28 smithi179 ceph-mon[36481]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:29.119 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:28 smithi092 ceph-mon[36049]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:29.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:28 smithi077 ceph-mon[32007]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:31.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:30 smithi092 ceph-mon[36049]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:31.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:30 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T08:33:31.088 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:30 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:31.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:30 smithi179 ceph-mon[36481]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:31.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:30 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T08:33:31.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:30 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:31.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:30 smithi077 ceph-mon[32007]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:31.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:30 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-27T08:33:31.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:30 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:31.910 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:31 smithi092 ceph-mon[36049]: Deploying daemon osd.10 on smithi092 2024-05-27T08:33:32.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:31 smithi179 ceph-mon[36481]: Deploying daemon osd.10 on smithi092 2024-05-27T08:33:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:31 smithi077 ceph-mon[32007]: Deploying daemon osd.10 on smithi092 2024-05-27T08:33:33.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:32 smithi179 ceph-mon[36481]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:33.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:32 smithi077 ceph-mon[32007]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:33.132 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:32 smithi092 ceph-mon[36049]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:35.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:34 smithi179 ceph-mon[36481]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:35.123 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:34 smithi092 ceph-mon[36049]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:35.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:34 smithi077 ceph-mon[32007]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:36 smithi092 ceph-mon[36049]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:36.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:36.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:36 smithi179 ceph-mon[36481]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:36 smithi077 ceph-mon[32007]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:39.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:38 smithi179 ceph-mon[36481]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:39.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:39.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:39.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.108 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:38 smithi092 ceph-mon[36049]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:39.108 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.108 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.109 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:39.109 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:39.109 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:38 smithi077 ceph-mon[32007]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:39.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:39.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:39.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:39.439 INFO:teuthology.orchestra.run.smithi092.stdout:Created osd(s) 10 on host 'smithi092' 2024-05-27T08:33:39.440 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.437+0000 7f59627fc700 1 -- 172.21.15.92:0/3275214005 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f596c06b9a0 con 0x7f59580b2880 2024-05-27T08:33:39.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f59580b2880 msgr2=0x7f59580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:39.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f59580b2880 0x7f59580b4d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f595c009bf0 tx=0x7f595c009260 comp rx=0 tx=0).stop 2024-05-27T08:33:39.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 msgr2=0x7f596c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:39.441 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 0x7f596c199820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f596800fa80 tx=0x7f596800fab0 comp rx=0 tx=0).stop 2024-05-27T08:33:39.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 shutdown_connections 2024-05-27T08:33:39.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f596c104040 0x7f596c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:39.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f596c1024a0 0x7f596c199820 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:39.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f59580b2880 0x7f59580b4d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:39.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 --2- 172.21.15.92:0/3275214005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f596c1036a0 0x7f596c199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:39.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 >> 172.21.15.92:0/3275214005 conn(0x7f596c0fda40 msgr2=0x7f596c06b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:33:39.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 shutdown_connections 2024-05-27T08:33:39.442 INFO:teuthology.orchestra.run.smithi092.stderr:2024-05-27T08:33:39.440+0000 7f5973265700 1 -- 172.21.15.92:0/3275214005 wait complete. 2024-05-27T08:33:39.973 DEBUG:teuthology.orchestra.run.smithi092:osd.10> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.10.service 2024-05-27T08:33:39.976 INFO:tasks.cephadm:Deploying osd.11 on smithi179 with /dev/vg_nvme/lv_1... 2024-05-27T08:33:39.976 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-27T08:33:40.222 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:33:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:40 smithi179 ceph-mon[36481]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:40 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:40 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:40 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:40.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:40 smithi179 ceph-mon[36481]: from='osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T08:33:40.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:40 smithi179 ceph-mon[36481]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T08:33:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:40 smithi092 ceph-mon[36049]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:40 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:40 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:40 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:40 smithi092 ceph-mon[36049]: from='osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T08:33:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:40 smithi092 ceph-mon[36049]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T08:33:40.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:40 smithi077 ceph-mon[32007]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:40.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:40 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:40.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:40 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:40.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:40 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:33:40.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:40 smithi077 ceph-mon[32007]: from='osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T08:33:40.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:40 smithi077 ceph-mon[32007]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-27T08:33:41.587 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:41 smithi179 ceph-mon[36481]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T08:33:41.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:41 smithi179 ceph-mon[36481]: from='osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:33:41.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:41 smithi179 ceph-mon[36481]: osdmap e66: 11 total, 10 up, 11 in 2024-05-27T08:33:41.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:41 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:41.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:41 smithi179 ceph-mon[36481]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:33:41.689 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:41 smithi092 ceph-mon[36049]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T08:33:41.689 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:41 smithi092 ceph-mon[36049]: from='osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:33:41.689 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:41 smithi092 ceph-mon[36049]: osdmap e66: 11 total, 10 up, 11 in 2024-05-27T08:33:41.689 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:41 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:41.689 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:41 smithi092 ceph-mon[36049]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:33:41.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:41 smithi077 ceph-mon[32007]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-27T08:33:41.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:41 smithi077 ceph-mon[32007]: from='osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:33:41.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:41 smithi077 ceph-mon[32007]: osdmap e66: 11 total, 10 up, 11 in 2024-05-27T08:33:41.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:41 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:41.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:41 smithi077 ceph-mon[32007]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]: dispatch 2024-05-27T08:33:42.152 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-27T08:33:42.152 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-27T08:33:42.152 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-05-27T08:33:42.152 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-05-27T08:33:42.152 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113972 s, 920 MB/s 2024-05-27T08:33:42.152 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-05-27T08:33:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:42 smithi092 ceph-mon[36049]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:42 smithi092 ceph-mon[36049]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:33:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:42 smithi092 ceph-mon[36049]: osdmap e67: 11 total, 10 up, 11 in 2024-05-27T08:33:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:42 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:42 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:42 smithi092 ceph-mon[36049]: from='osd.10 ' entity='osd.10' 2024-05-27T08:33:42.737 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:33:42 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:33:42.350+0000 7f84a1523700 -1 osd.10 0 waiting for initial osdmap 2024-05-27T08:33:42.737 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:33:42 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:33:42.528+0000 7f849930e700 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:42 smithi179 ceph-mon[36481]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:42 smithi179 ceph-mon[36481]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:42 smithi179 ceph-mon[36481]: osdmap e67: 11 total, 10 up, 11 in 2024-05-27T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:42 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:42 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:42 smithi179 ceph-mon[36481]: from='osd.10 ' entity='osd.10' 2024-05-27T08:33:42.773 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2024-05-27T08:33:42.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:42 smithi077 ceph-mon[32007]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:42.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:42 smithi077 ceph-mon[32007]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi092", "root=default"]}]': finished 2024-05-27T08:33:42.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:42 smithi077 ceph-mon[32007]: osdmap e67: 11 total, 10 up, 11 in 2024-05-27T08:33:42.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:42 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:42.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:42 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:42.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:42 smithi077 ceph-mon[32007]: from='osd.10 ' entity='osd.10' 2024-05-27T08:33:43.015 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:33:43.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:33:43.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:33:43.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: osdmap e68: 11 total, 10 up, 11 in 2024-05-27T08:33:43.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:43.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:43 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.820 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:33:43.820 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:33:43.820 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: osdmap e68: 11 total, 10 up, 11 in 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:43.821 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:43 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: osdmap e68: 11 total, 10 up, 11 in 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:33:43.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:43.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:43.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:33:43.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:43 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:33:44.547 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 -- 172.21.15.179:0/4016413190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 msgr2=0x7f426810a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 --2- 172.21.15.179:0/4016413190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 0x7f426810a980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4260009990 tx=0x7f4260009ca0 comp rx=0 tx=0).stop 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 -- 172.21.15.179:0/4016413190 shutdown_connections 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 --2- 172.21.15.179:0/4016413190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 0x7f426810a980 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 --2- 172.21.15.179:0/4016413190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268103a70 0x7f4268103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 --2- 172.21.15.179:0/4016413190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4268102720 0x7f4268102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 -- 172.21.15.179:0/4016413190 >> 172.21.15.179:0/4016413190 conn(0x7f42680fd1d0 msgr2=0x7f42680ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 -- 172.21.15.179:0/4016413190 shutdown_connections 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 -- 172.21.15.179:0/4016413190 wait complete. 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.546+0000 7f426d79c700 1 Processor -- start 2024-05-27T08:33:44.548 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f426d79c700 1 -- start start 2024-05-27T08:33:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f426d79c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4268102720 0x7f42681064c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:33:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f426d79c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 0x7f4268104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:33:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f426d79c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268103a70 0x7f4268105050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:33:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f426d79c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f426810df20 con 0x7f4268102720 2024-05-27T08:33:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f426d79c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f426810dda0 con 0x7f4268103a70 2024-05-27T08:33:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f426d79c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f426810e0a0 con 0x7f42681030c0 2024-05-27T08:33:44.550 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 0x7f4268104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 0x7f4268104b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37050/0 (socket says 172.21.15.179:37050) 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 -- 172.21.15.179:0/3025044369 learned_addr learned my addr 172.21.15.179:0/3025044369 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f42677fe700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268103a70 0x7f4268105050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4266ffd700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4268102720 0x7f42681064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 -- 172.21.15.179:0/3025044369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268103a70 msgr2=0x7f4268105050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268103a70 0x7f4268105050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 -- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4268102720 msgr2=0x7f42681064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4268102720 0x7f42681064c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 -- 172.21.15.179:0/3025044369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4268105900 con 0x7f42681030c0 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4266ffd700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4268102720 0x7f42681064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4257fff700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 0x7f4268104b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f425c00d950 tx=0x7f425c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4264ff9700 1 -- 172.21.15.179:0/3025044369 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c016070 con 0x7f42681030c0 2024-05-27T08:33:44.551 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4264ff9700 1 -- 172.21.15.179:0/3025044369 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f425c009df0 con 0x7f42681030c0 2024-05-27T08:33:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42681a3860 con 0x7f42681030c0 2024-05-27T08:33:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.547+0000 7f4264ff9700 1 -- 172.21.15.179:0/3025044369 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c00f3c0 con 0x7f42681030c0 2024-05-27T08:33:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.548+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42681a3d70 con 0x7f42681030c0 2024-05-27T08:33:44.552 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.548+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426806a230 con 0x7f42681030c0 2024-05-27T08:33:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.550+0000 7f4264ff9700 1 -- 172.21.15.179:0/3025044369 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f425c019070 con 0x7f42681030c0 2024-05-27T08:33:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.550+0000 7f4264ff9700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42500b2880 0x7f42500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:33:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.551+0000 7f4266ffd700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42500b2880 0x7f42500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:33:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.551+0000 7f4264ff9700 1 -- 172.21.15.179:0/3025044369 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6694+0+0 (secure 0 0 0) 0x7f425c0e17f0 con 0x7f42681030c0 2024-05-27T08:33:44.553 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.551+0000 7f4266ffd700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42500b2880 0x7f42500b4d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f426806afe0 tx=0x7f4258009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:33:44.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.554+0000 7f4264ff9700 1 -- 172.21.15.179:0/3025044369 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f425c0a9b40 con 0x7f42681030c0 2024-05-27T08:33:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:44 smithi179 ceph-mon[36481]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:44 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi092 2024-05-27T08:33:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:44 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-27T08:33:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:44 smithi179 ceph-mon[36481]: osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] boot 2024-05-27T08:33:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:44 smithi179 ceph-mon[36481]: osdmap e69: 11 total, 11 up, 11 in 2024-05-27T08:33:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:44.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:44 smithi092 ceph-mon[36049]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:44.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:44 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi092 2024-05-27T08:33:44.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:44 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-27T08:33:44.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:44.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:44 smithi092 ceph-mon[36049]: osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] boot 2024-05-27T08:33:44.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:44 smithi092 ceph-mon[36049]: osdmap e69: 11 total, 11 up, 11 in 2024-05-27T08:33:44.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:44.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:33:44.767+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f42680ff340 con 0x7f42500b2880 2024-05-27T08:33:44.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:44 smithi077 ceph-mon[32007]: pgmap v231: 1 pgs: 1 peering; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-05-27T08:33:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:44 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi092 2024-05-27T08:33:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:44 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi092 to 4272M 2024-05-27T08:33:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:44 smithi077 ceph-mon[32007]: osd.10 [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] boot 2024-05-27T08:33:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:44 smithi077 ceph-mon[32007]: osdmap e69: 11 total, 11 up, 11 in 2024-05-27T08:33:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:33:45.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:45 smithi179 ceph-mon[36481]: osdmap e70: 11 total, 11 up, 11 in 2024-05-27T08:33:45.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:45 smithi179 ceph-mon[36481]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:33:45.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:45 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:33:45.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:45 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:33:45.686 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:45 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:45.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:45 smithi092 ceph-mon[36049]: osdmap e70: 11 total, 11 up, 11 in 2024-05-27T08:33:45.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:45 smithi092 ceph-mon[36049]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:33:45.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:45 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:33:45.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:45 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:33:45.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:45 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:45.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:45 smithi077 ceph-mon[32007]: osdmap e70: 11 total, 11 up, 11 in 2024-05-27T08:33:45.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:45 smithi077 ceph-mon[32007]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:33:45.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:45 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-27T08:33:45.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:45 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-27T08:33:45.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:45 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:46.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:46 smithi179 ceph-mon[36481]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:33:46.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:46 smithi179 ceph-mon[36481]: osdmap e71: 11 total, 11 up, 11 in 2024-05-27T08:33:46.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:46 smithi092 ceph-mon[36049]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:33:46.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:46 smithi092 ceph-mon[36049]: osdmap e71: 11 total, 11 up, 11 in 2024-05-27T08:33:46.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:46 smithi077 ceph-mon[32007]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:33:46.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:46 smithi077 ceph-mon[32007]: osdmap e71: 11 total, 11 up, 11 in 2024-05-27T08:33:48.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:48 smithi179 ceph-mon[36481]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-27T08:33:48.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:48 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/662579735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]: dispatch 2024-05-27T08:33:48.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:48 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]: dispatch 2024-05-27T08:33:48.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:48 smithi179 ceph-mon[36481]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]': finished 2024-05-27T08:33:48.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:48 smithi179 ceph-mon[36481]: osdmap e72: 12 total, 11 up, 12 in 2024-05-27T08:33:48.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:33:48.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:48 smithi179 ceph-mon[36481]: from='client.? 172.21.15.179:0/3048746534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:33:48.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:48 smithi077 ceph-mon[32007]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-27T08:33:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:48 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/662579735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]: dispatch 2024-05-27T08:33:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:48 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]: dispatch 2024-05-27T08:33:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:48 smithi077 ceph-mon[32007]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]': finished 2024-05-27T08:33:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:48 smithi077 ceph-mon[32007]: osdmap e72: 12 total, 11 up, 12 in 2024-05-27T08:33:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:33:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:48 smithi077 ceph-mon[32007]: from='client.? 172.21.15.179:0/3048746534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:33:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:48 smithi092 ceph-mon[36049]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-05-27T08:33:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:48 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/662579735' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]: dispatch 2024-05-27T08:33:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:48 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]: dispatch 2024-05-27T08:33:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:48 smithi092 ceph-mon[36049]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b"}]': finished 2024-05-27T08:33:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:48 smithi092 ceph-mon[36049]: osdmap e72: 12 total, 11 up, 12 in 2024-05-27T08:33:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:33:48.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:48 smithi092 ceph-mon[36049]: from='client.? 172.21.15.179:0/3048746534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-27T08:33:50.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:50 smithi179 ceph-mon[36481]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-27T08:33:50.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:50 smithi077 ceph-mon[32007]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-27T08:33:50.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:50 smithi092 ceph-mon[36049]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-05-27T08:33:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:52 smithi179 ceph-mon[36481]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-27T08:33:52.829 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:52 smithi092 ceph-mon[36049]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-27T08:33:52.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:52 smithi077 ceph-mon[32007]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-05-27T08:33:54.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:54 smithi179 ceph-mon[36481]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:33:54.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:54 smithi077 ceph-mon[32007]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:33:54.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:54 smithi092 ceph-mon[36049]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-27T08:33:55.796 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:55 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T08:33:55.796 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:55 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:55.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:55 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T08:33:55.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:55 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:55.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:55 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-27T08:33:55.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:55 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:33:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:56 smithi179 ceph-mon[36481]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:33:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:56 smithi179 ceph-mon[36481]: Deploying daemon osd.11 on smithi179 2024-05-27T08:33:56.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:56 smithi077 ceph-mon[32007]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:33:56.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:56 smithi077 ceph-mon[32007]: Deploying daemon osd.11 on smithi179 2024-05-27T08:33:56.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:56 smithi092 ceph-mon[36049]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-27T08:33:56.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:56 smithi092 ceph-mon[36049]: Deploying daemon osd.11 on smithi179 2024-05-27T08:33:58.822 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:33:58 smithi179 ceph-mon[36481]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:33:58.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:33:58 smithi077 ceph-mon[32007]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:33:58.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:33:58 smithi092 ceph-mon[36049]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:00.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:00 smithi179 ceph-mon[36481]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:00.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:00 smithi077 ceph-mon[32007]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:00.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:00 smithi092 ceph-mon[36049]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:01.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:01.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:01.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:01 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:01.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:01.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:01.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:01 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:02.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:02.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:02.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:01 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:02.900 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:02 smithi179 ceph-mon[36481]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:02.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:02 smithi092 ceph-mon[36049]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:03.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:02 smithi077 ceph-mon[32007]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:04.684 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.683+0000 7f4264ff9700 1 -- 172.21.15.179:0/3025044369 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f42680ff340 con 0x7f42500b2880 2024-05-27T08:34:04.685 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 11 on host 'smithi179' 2024-05-27T08:34:04.686 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42500b2880 msgr2=0x7f42500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42500b2880 0x7f42500b4d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f426806afe0 tx=0x7f4258009040 comp rx=0 tx=0).stop 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 msgr2=0x7f4268104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 0x7f4268104b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f425c00d950 tx=0x7f425c00dc60 comp rx=0 tx=0).stop 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 shutdown_connections 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4268103a70 0x7f4268105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4268102720 0x7f42681064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42500b2880 0x7f42500b4d40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 --2- 172.21.15.179:0/3025044369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42681030c0 0x7f4268104b10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 >> 172.21.15.179:0/3025044369 conn(0x7f42680fd1d0 msgr2=0x7f42680fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 shutdown_connections 2024-05-27T08:34:04.687 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:04.685+0000 7f426d79c700 1 -- 172.21.15.179:0/3025044369 wait complete. 2024-05-27T08:34:04.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:04.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:04.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:04.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:04.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:04.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:04.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T08:34:04.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T08:34:04.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:04.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:04.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:04 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:05.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:05.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:05.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:05.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T08:34:05.132 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T08:34:05.132 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.132 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.132 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:04 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T08:34:05.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-27T08:34:05.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:05.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:04 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:05.326 DEBUG:teuthology.orchestra.run.smithi179:osd.11> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.11.service 2024-05-27T08:34:05.329 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-27T08:34:05.329 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-05-27T08:34:05.580 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:06.007 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:05 smithi077 ceph-mon[32007]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T08:34:06.007 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:05 smithi077 ceph-mon[32007]: osdmap e73: 12 total, 11 up, 12 in 2024-05-27T08:34:06.007 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:06.007 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:05 smithi077 ceph-mon[32007]: from='osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:34:06.008 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:05 smithi077 ceph-mon[32007]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:34:06.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:05 smithi179 ceph-mon[36481]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T08:34:06.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:05 smithi179 ceph-mon[36481]: osdmap e73: 12 total, 11 up, 12 in 2024-05-27T08:34:06.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:06.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:05 smithi179 ceph-mon[36481]: from='osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:34:06.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:05 smithi179 ceph-mon[36481]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:34:06.108 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:34:05 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:34:05.816+0000 7f4532787700 -1 osd.11 0 waiting for initial osdmap 2024-05-27T08:34:06.108 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:34:05 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:34:05.829+0000 7f452e57a700 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-27T08:34:06.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:05 smithi092 ceph-mon[36049]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-27T08:34:06.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:05 smithi092 ceph-mon[36049]: osdmap e73: 12 total, 11 up, 12 in 2024-05-27T08:34:06.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:06.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:05 smithi092 ceph-mon[36049]: from='osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:34:06.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:05 smithi092 ceph-mon[36049]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-05-27T08:34:07.000 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:06 smithi077 ceph-mon[32007]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:07.000 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:06 smithi077 ceph-mon[32007]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:34:07.000 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:06 smithi077 ceph-mon[32007]: osdmap e74: 12 total, 11 up, 12 in 2024-05-27T08:34:07.000 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:06 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:07.000 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:06 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:06 smithi179 ceph-mon[36481]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:06 smithi179 ceph-mon[36481]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:34:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:06 smithi179 ceph-mon[36481]: osdmap e74: 12 total, 11 up, 12 in 2024-05-27T08:34:07.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:06 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:07.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:06 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:07.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.207+0000 7f3d858bd700 1 -- 172.21.15.77:0/3583890174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80104030 msgr2=0x7f3d8010a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.207+0000 7f3d858bd700 1 --2- 172.21.15.77:0/3583890174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80104030 0x7f3d8010a8e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800b0a0 tx=0x7f3d7800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 -- 172.21.15.77:0/3583890174 shutdown_connections 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 --2- 172.21.15.77:0/3583890174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80104030 0x7f3d8010a8e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 --2- 172.21.15.77:0/3583890174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d80102490 0x7f3d801028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 --2- 172.21.15.77:0/3583890174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d80103690 0x7f3d80103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 -- 172.21.15.77:0/3583890174 >> 172.21.15.77:0/3583890174 conn(0x7f3d800fda30 msgr2=0x7f3d800ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 -- 172.21.15.77:0/3583890174 shutdown_connections 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 -- 172.21.15.77:0/3583890174 wait complete. 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 Processor -- start 2024-05-27T08:34:07.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 -- start start 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d80102490 0x7f3d8019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.208+0000 7f3d858bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80103690 0x7f3d8019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d858bd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d80104030 0x7f3d801a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d858bd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010f0b0 con 0x7f3d80103690 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d858bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010ef30 con 0x7f3d80102490 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d858bd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8010f230 con 0x7f3d80104030 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80103690 0x7f3d8019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80103690 0x7f3d8019bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:34492/0 (socket says 172.21.15.77:34492) 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 -- 172.21.15.77:0/746565058 learned_addr learned my addr 172.21.15.77:0/746565058 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7f7fe700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d80104030 0x7f3d801a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7effd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d80102490 0x7f3d8019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 -- 172.21.15.77:0/746565058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d80104030 msgr2=0x7f3d801a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:07.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d80104030 0x7f3d801a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 -- 172.21.15.77:0/746565058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d80102490 msgr2=0x7f3d8019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d80102490 0x7f3d8019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 -- 172.21.15.77:0/746565058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d801a3500 con 0x7f3d80103690 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d7e7fc700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80103690 0x7f3d8019bde0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400b560 tx=0x7f3d7400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d848bb700 1 -- 172.21.15.77:0/746565058 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74015020 con 0x7f3d80103690 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d848bb700 1 -- 172.21.15.77:0/746565058 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d74011ce0 con 0x7f3d80103690 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d801a37f0 con 0x7f3d80103690 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d848bb700 1 -- 172.21.15.77:0/746565058 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74010d20 con 0x7f3d80103690 2024-05-27T08:34:07.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.209+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d801a3d30 con 0x7f3d80103690 2024-05-27T08:34:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.210+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8006a230 con 0x7f3d80103690 2024-05-27T08:34:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.212+0000 7f3d848bb700 1 -- 172.21.15.77:0/746565058 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3d74018020 con 0x7f3d80103690 2024-05-27T08:34:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.212+0000 7f3d848bb700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3d680b2880 0x7f3d680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.212+0000 7f3d7effd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3d680b2880 0x7f3d680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.212+0000 7f3d848bb700 1 -- 172.21.15.77:0/746565058 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f3d740e14d0 con 0x7f3d80103690 2024-05-27T08:34:07.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.213+0000 7f3d7effd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3d680b2880 0x7f3d680b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d801034f0 tx=0x7f3d7000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:07.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.217+0000 7f3d848bb700 1 -- 172.21.15.77:0/746565058 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d7401f020 con 0x7f3d80103690 2024-05-27T08:34:07.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:06 smithi092 ceph-mon[36049]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:07.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:06 smithi092 ceph-mon[36049]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-05-27T08:34:07.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:06 smithi092 ceph-mon[36049]: osdmap e74: 12 total, 11 up, 12 in 2024-05-27T08:34:07.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:06 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:07.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:06 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:07.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.423+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3d801a41b0 con 0x7f3d80103690 2024-05-27T08:34:07.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.424+0000 7f3d848bb700 1 -- 172.21.15.77:0/746565058 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f3d740a9690 con 0x7f3d80103690 2024-05-27T08:34:07.426 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3d680b2880 msgr2=0x7f3d680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3d680b2880 0x7f3d680b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3d801034f0 tx=0x7f3d7000b5b0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80103690 msgr2=0x7f3d8019bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80103690 0x7f3d8019bde0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400b560 tx=0x7f3d7400b870 comp rx=0 tx=0).stop 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 shutdown_connections 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3d680b2880 0x7f3d680b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d80104030 0x7f3d801a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3d80102490 0x7f3d8019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 --2- 172.21.15.77:0/746565058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3d80103690 0x7f3d8019bde0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:07.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.426+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 >> 172.21.15.77:0/746565058 conn(0x7f3d800fda30 msgr2=0x7f3d8010b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:07.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.427+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 shutdown_connections 2024-05-27T08:34:07.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:07.427+0000 7f3d858bd700 1 -- 172.21.15.77:0/746565058 wait complete. 2024-05-27T08:34:07.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:07 smithi179 ceph-mon[36481]: purged_snaps scrub starts 2024-05-27T08:34:07.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:07 smithi179 ceph-mon[36481]: purged_snaps scrub ok 2024-05-27T08:34:07.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:07 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:07.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:07 smithi179 ceph-mon[36481]: osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] boot 2024-05-27T08:34:07.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:07 smithi179 ceph-mon[36481]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T08:34:07.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:07 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:07.988 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:07 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/746565058' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T08:34:08.035 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:07 smithi077 ceph-mon[32007]: purged_snaps scrub starts 2024-05-27T08:34:08.035 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:07 smithi077 ceph-mon[32007]: purged_snaps scrub ok 2024-05-27T08:34:08.036 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:07 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:08.036 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:07 smithi077 ceph-mon[32007]: osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] boot 2024-05-27T08:34:08.036 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:07 smithi077 ceph-mon[32007]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T08:34:08.036 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:07 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:08.036 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:07 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/746565058' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T08:34:08.063 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1716798846,"num_in_osds":12,"osd_in_since":1716798827,"num_remapped_pgs":0} 2024-05-27T08:34:08.064 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:34:08.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:07 smithi092 ceph-mon[36049]: purged_snaps scrub starts 2024-05-27T08:34:08.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:07 smithi092 ceph-mon[36049]: purged_snaps scrub ok 2024-05-27T08:34:08.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:07 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:08.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:07 smithi092 ceph-mon[36049]: osd.11 [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] boot 2024-05-27T08:34:08.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:07 smithi092 ceph-mon[36049]: osdmap e75: 12 total, 12 up, 12 in 2024-05-27T08:34:08.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:07 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:34:08.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:07 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/746565058' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-27T08:34:08.311 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:09.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:09.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:08 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:09.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:09.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:08 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-05-27T08:34:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: osdmap e76: 12 total, 12 up, 12 in 2024-05-27T08:34:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:34:09.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:09.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:09.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:09.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:08 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:10.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 -- 172.21.15.77:0/45776322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0108920 msgr2=0x7ff3b01104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:10.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 --2- 172.21.15.77:0/45776322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0108920 0x7ff3b01104a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800b0d0 tx=0x7ff3a800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 -- 172.21.15.77:0/45776322 shutdown_connections 2024-05-27T08:34:10.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 --2- 172.21.15.77:0/45776322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0108920 0x7ff3b01104a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 --2- 172.21.15.77:0/45776322 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0103850 0x7ff3b0103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.097 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 --2- 172.21.15.77:0/45776322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b0104200 0x7ff3b01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 -- 172.21.15.77:0/45776322 >> 172.21.15.77:0/45776322 conn(0x7ff3b007a5d0 msgr2=0x7ff3b007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:10.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 -- 172.21.15.77:0/45776322 shutdown_connections 2024-05-27T08:34:10.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.095+0000 7ff3b498d700 1 -- 172.21.15.77:0/45776322 wait complete. 2024-05-27T08:34:10.098 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:09 smithi077 ceph-mon[32007]: Detected new or changed devices on smithi179 2024-05-27T08:34:10.098 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:09 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi179 to 4272M 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.096+0000 7ff3b498d700 1 Processor -- start 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.096+0000 7ff3b498d700 1 -- start start 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.096+0000 7ff3b498d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0103850 0x7ff3b0074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.096+0000 7ff3b498d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b0104200 0x7ff3b00750b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.096+0000 7ff3b498d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0108920 0x7ff3b006dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.096+0000 7ff3b498d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0113710 con 0x7ff3b0103850 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.096+0000 7ff3b498d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0113590 con 0x7ff3b0108920 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.096+0000 7ff3b498d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0113890 con 0x7ff3b0104200 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0103850 0x7ff3b0074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0103850 0x7ff3b0074b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44922/0 (socket says 172.21.15.77:44922) 2024-05-27T08:34:10.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 -- 172.21.15.77:0/3674010386 learned_addr learned my addr 172.21.15.77:0/3674010386 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3add9b700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b0104200 0x7ff3b00750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3aed9d700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0108920 0x7ff3b006dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 -- 172.21.15.77:0/3674010386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b0104200 msgr2=0x7ff3b00750b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b0104200 0x7ff3b00750b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 -- 172.21.15.77:0/3674010386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0108920 msgr2=0x7ff3b006dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0108920 0x7ff3b006dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 -- 172.21.15.77:0/3674010386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b006e5d0 con 0x7ff3b0103850 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3aed9d700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0108920 0x7ff3b006dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3ae59c700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0103850 0x7ff3b0074b70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c9b0 tx=0x7ff3a400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff39f7fe700 1 -- 172.21.15.77:0/3674010386 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4007b00 con 0x7ff3b0103850 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b006e8c0 con 0x7ff3b0103850 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff39f7fe700 1 -- 172.21.15.77:0/3674010386 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a4007ca0 con 0x7ff3b0103850 2024-05-27T08:34:10.101 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff39f7fe700 1 -- 172.21.15.77:0/3674010386 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4018660 con 0x7ff3b0103850 2024-05-27T08:34:10.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.097+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b00728c0 con 0x7ff3b0103850 2024-05-27T08:34:10.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.098+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b0104cb0 con 0x7ff3b0103850 2024-05-27T08:34:10.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.100+0000 7ff39f7fe700 1 -- 172.21.15.77:0/3674010386 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff3a400f450 con 0x7ff3b0103850 2024-05-27T08:34:10.102 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.100+0000 7ff39f7fe700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff3980b2880 0x7ff3980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:10.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.100+0000 7ff3add9b700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff3980b2880 0x7ff3980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:10.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.100+0000 7ff39f7fe700 1 -- 172.21.15.77:0/3674010386 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7ff3a40e1d50 con 0x7ff3b0103850 2024-05-27T08:34:10.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.101+0000 7ff3add9b700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff3980b2880 0x7ff3980b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3b006f3c0 tx=0x7ff3a000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:10.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.105+0000 7ff39f7fe700 1 -- 172.21.15.77:0/3674010386 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3a40e5030 con 0x7ff3b0103850 2024-05-27T08:34:10.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:09 smithi179 ceph-mon[36481]: Detected new or changed devices on smithi179 2024-05-27T08:34:10.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:09 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi179 to 4272M 2024-05-27T08:34:10.137 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:09 smithi092 ceph-mon[36049]: Detected new or changed devices on smithi179 2024-05-27T08:34:10.137 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:09 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi179 to 4272M 2024-05-27T08:34:10.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.312+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3b0072bf0 con 0x7ff3b0103850 2024-05-27T08:34:10.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.313+0000 7ff39f7fe700 1 -- 172.21.15.77:0/3674010386 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18432 (secure 0 0 0) 0x7ff3a4014030 con 0x7ff3b0103850 2024-05-27T08:34:10.315 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:10.315 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":76,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:34:07.822661+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:34:10.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff3980b2880 msgr2=0x7ff3980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:10.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff3980b2880 0x7ff3980b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3b006f3c0 tx=0x7ff3a000b3c0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0103850 msgr2=0x7ff3b0074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:10.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0103850 0x7ff3b0074b70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400c9b0 tx=0x7ff3a400ccc0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 shutdown_connections 2024-05-27T08:34:10.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b0108920 0x7ff3b006dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b0103850 0x7ff3b0074b70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff3980b2880 0x7ff3980b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 --2- 172.21.15.77:0/3674010386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b0104200 0x7ff3b00750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:10.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 >> 172.21.15.77:0/3674010386 conn(0x7ff3b007a5d0 msgr2=0x7ff3b0101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:10.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 shutdown_connections 2024-05-27T08:34:10.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:10.316+0000 7ff3b498d700 1 -- 172.21.15.77:0/3674010386 wait complete. 2024-05-27T08:34:10.983 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:10 smithi077 ceph-mon[32007]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:10.983 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:10 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3674010386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:34:11.011 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T08:30:25.158063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T08:34:11.012 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-27T08:34:11.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:10 smithi179 ceph-mon[36481]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:11.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:10 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3674010386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:34:11.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:10 smithi092 ceph-mon[36049]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:11.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:10 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3674010386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:34:11.249 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:12.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/3686283102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb88102470 msgr2=0x7fdb881028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/3686283102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb88102470 0x7fdb881028f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009990 tx=0x7fdb78009ca0 comp rx=0 tx=0).stop 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/3686283102 shutdown_connections 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/3686283102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb88102e30 0x7fdb8810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/3686283102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb881084b0 0x7fdb88108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/3686283102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb88102470 0x7fdb881028f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/3686283102 >> 172.21.15.77:0/3686283102 conn(0x7fdb880793f0 msgr2=0x7fdb88079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/3686283102 shutdown_connections 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.813+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/3686283102 wait complete. 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8f2d9700 1 Processor -- start 2024-05-27T08:34:12.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8f2d9700 1 -- start start 2024-05-27T08:34:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8f2d9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102470 0x7fdb88104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8f2d9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb88102e30 0x7fdb881045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8f2d9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb881084b0 0x7fdb88104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8f2d9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb881125e0 con 0x7fdb881084b0 2024-05-27T08:34:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8f2d9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88112460 con 0x7fdb88102e30 2024-05-27T08:34:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8f2d9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb88112760 con 0x7fdb88102470 2024-05-27T08:34:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb881084b0 0x7fdb88104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:12.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d075700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102470 0x7fdb88104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb881084b0 0x7fdb88104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44950/0 (socket says 172.21.15.77:44950) 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 -- 172.21.15.77:0/4126502938 learned_addr learned my addr 172.21.15.77:0/4126502938 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8c874700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb88102e30 0x7fdb881045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 -- 172.21.15.77:0/4126502938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102470 msgr2=0x7fdb88104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102470 0x7fdb88104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 -- 172.21.15.77:0/4126502938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb88102e30 msgr2=0x7fdb881045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb88102e30 0x7fdb881045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 -- 172.21.15.77:0/4126502938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb881a8170 con 0x7fdb881084b0 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.814+0000 7fdb8d876700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb881084b0 0x7fdb88104b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000ada0 tx=0x7fdb8000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.815+0000 7fdb7e7fc700 1 -- 172.21.15.77:0/4126502938 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80017020 con 0x7fdb881084b0 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.815+0000 7fdb7e7fc700 1 -- 172.21.15.77:0/4126502938 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8000cd80 con 0x7fdb881084b0 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.815+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb881a8460 con 0x7fdb881084b0 2024-05-27T08:34:12.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.815+0000 7fdb7e7fc700 1 -- 172.21.15.77:0/4126502938 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80005260 con 0x7fdb881084b0 2024-05-27T08:34:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.815+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb881a8920 con 0x7fdb881084b0 2024-05-27T08:34:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.816+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb88103b60 con 0x7fdb881084b0 2024-05-27T08:34:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.817+0000 7fdb7e7fc700 1 -- 172.21.15.77:0/4126502938 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdb8000a090 con 0x7fdb881084b0 2024-05-27T08:34:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.818+0000 7fdb7e7fc700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdb740b2880 0x7fdb740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:12.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.818+0000 7fdb8d075700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdb740b2880 0x7fdb740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:12.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.818+0000 7fdb7e7fc700 1 -- 172.21.15.77:0/4126502938 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fdb800e1a80 con 0x7fdb881084b0 2024-05-27T08:34:12.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.818+0000 7fdb8d075700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdb740b2880 0x7fdb740b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb840045b0 tx=0x7fdb8400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:12.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:12.821+0000 7fdb7e7fc700 1 -- 172.21.15.77:0/4126502938 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdb80021020 con 0x7fdb881084b0 2024-05-27T08:34:13.026 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.025+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb8802e560 con 0x7fdb881084b0 2024-05-27T08:34:13.026 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:12 smithi077 ceph-mon[32007]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:13.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.025+0000 7fdb7e7fc700 1 -- 172.21.15.77:0/4126502938 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fdb800a9cd0 con 0x7fdb881084b0 2024-05-27T08:34:13.027 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-05-27T08:34:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdb740b2880 msgr2=0x7fdb740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdb740b2880 0x7fdb740b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb840045b0 tx=0x7fdb8400b3c0 comp rx=0 tx=0).stop 2024-05-27T08:34:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb881084b0 msgr2=0x7fdb88104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:13.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb881084b0 0x7fdb88104b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000ada0 tx=0x7fdb8000c330 comp rx=0 tx=0).stop 2024-05-27T08:34:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 shutdown_connections 2024-05-27T08:34:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb881084b0 0x7fdb88104b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdb740b2880 0x7fdb740b4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102470 0x7fdb88104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.027+0000 7fdb8f2d9700 1 --2- 172.21.15.77:0/4126502938 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdb88102e30 0x7fdb881045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.028+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 >> 172.21.15.77:0/4126502938 conn(0x7fdb880793f0 msgr2=0x7fdb8806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.028+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 shutdown_connections 2024-05-27T08:34:13.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:13.028+0000 7fdb8f2d9700 1 -- 172.21.15.77:0/4126502938 wait complete. 2024-05-27T08:34:13.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:12 smithi179 ceph-mon[36481]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:13.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:12 smithi092 ceph-mon[36049]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:13.658 INFO:tasks.cephadm:Adding mds.a on smithi077 2024-05-27T08:34:13.658 INFO:tasks.cephadm:Adding mds.d on smithi077 2024-05-27T08:34:13.658 INFO:tasks.cephadm:Adding mds.g on smithi077 2024-05-27T08:34:13.658 INFO:tasks.cephadm:Adding mds.j on smithi077 2024-05-27T08:34:13.658 INFO:tasks.cephadm:Adding mds.b on smithi092 2024-05-27T08:34:13.659 INFO:tasks.cephadm:Adding mds.e on smithi092 2024-05-27T08:34:13.659 INFO:tasks.cephadm:Adding mds.h on smithi092 2024-05-27T08:34:13.659 INFO:tasks.cephadm:Adding mds.k on smithi092 2024-05-27T08:34:13.659 INFO:tasks.cephadm:Adding mds.c on smithi179 2024-05-27T08:34:13.659 INFO:tasks.cephadm:Adding mds.f on smithi179 2024-05-27T08:34:13.659 INFO:tasks.cephadm:Adding mds.i on smithi179 2024-05-27T08:34:13.659 INFO:tasks.cephadm:Adding mds.l on smithi179 2024-05-27T08:34:13.659 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l' 2024-05-27T08:34:13.909 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:34:14.031 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:13 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/4126502938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:34:14.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:13 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/4126502938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:34:14.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:13 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/4126502938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:34:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:14 smithi179 ceph-mon[36481]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:15.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:14 smithi077 ceph-mon[32007]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:14 smithi092 ceph-mon[36049]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:15.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 -- 172.21.15.179:0/207167174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30103b70 msgr2=0x7f9c30103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:15.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 --2- 172.21.15.179:0/207167174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30103b70 0x7f9c30103f90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009990 tx=0x7f9c24009ca0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 -- 172.21.15.179:0/207167174 shutdown_connections 2024-05-27T08:34:15.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 --2- 172.21.15.179:0/207167174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c30103100 0x7f9c3010a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.424 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 --2- 172.21.15.179:0/207167174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30103b70 0x7f9c30103f90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 --2- 172.21.15.179:0/207167174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c30102740 0x7f9c30102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 -- 172.21.15.179:0/207167174 >> 172.21.15.179:0/207167174 conn(0x7f9c300fd210 msgr2=0x7f9c300ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 -- 172.21.15.179:0/207167174 shutdown_connections 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.421+0000 7f9c36efa700 1 -- 172.21.15.179:0/207167174 wait complete. 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c36efa700 1 Processor -- start 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c36efa700 1 -- start start 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c36efa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30102740 0x7f9c30102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c36efa700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c30103100 0x7f9c3019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c36efa700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c30103b70 0x7f9c301a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c36efa700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3010dfc0 con 0x7f9c30103100 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c36efa700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3010de40 con 0x7f9c30103b70 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c36efa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3010e140 con 0x7f9c30102740 2024-05-27T08:34:15.425 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c34c96700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30102740 0x7f9c30102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c34c96700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30102740 0x7f9c30102190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:59398/0 (socket says 172.21.15.179:59398) 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c34c96700 1 -- 172.21.15.179:0/935519744 learned_addr learned my addr 172.21.15.179:0/935519744 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c34c96700 1 -- 172.21.15.179:0/935519744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c30103b70 msgr2=0x7f9c301a0a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c34c96700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c30103b70 0x7f9c301a0a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c34c96700 1 -- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c30103100 msgr2=0x7f9c3019e440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c34c96700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c30103100 0x7f9c3019e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.422+0000 7f9c34c96700 1 -- 172.21.15.179:0/935519744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c301a1350 con 0x7f9c30102740 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.423+0000 7f9c34c96700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30102740 0x7f9c30102190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9c240036f0 tx=0x7f9c240039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.423+0000 7f9c2dffb700 1 -- 172.21.15.179:0/935519744 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c24027070 con 0x7f9c30102740 2024-05-27T08:34:15.426 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.423+0000 7f9c2dffb700 1 -- 172.21.15.179:0/935519744 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c24022470 con 0x7f9c30102740 2024-05-27T08:34:15.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.423+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c301a15e0 con 0x7f9c30102740 2024-05-27T08:34:15.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.423+0000 7f9c2dffb700 1 -- 172.21.15.179:0/935519744 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2400f6c0 con 0x7f9c30102740 2024-05-27T08:34:15.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.423+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c301a1a90 con 0x7f9c30102740 2024-05-27T08:34:15.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.424+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c3006a230 con 0x7f9c30102740 2024-05-27T08:34:15.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.426+0000 7f9c2dffb700 1 -- 172.21.15.179:0/935519744 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9c2402a070 con 0x7f9c30102740 2024-05-27T08:34:15.427 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.427+0000 7f9c2dffb700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9c200b2900 0x7f9c200b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:15.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.427+0000 7f9c2dffb700 1 -- 172.21.15.179:0/935519744 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f9c240e3130 con 0x7f9c30102740 2024-05-27T08:34:15.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.427+0000 7f9c2ffff700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9c200b2900 0x7f9c200b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:15.428 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.427+0000 7f9c2ffff700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9c200b2900 0x7f9c200b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c30102710 tx=0x7f9c1c005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:15.431 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.430+0000 7f9c2dffb700 1 -- 172.21.15.179:0/935519744 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c24013070 con 0x7f9c30102740 2024-05-27T08:34:15.645 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.643+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}) -- 0x7f9c30063c40 con 0x7f9c200b2900 2024-05-27T08:34:15.653 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.652+0000 7f9c2dffb700 1 -- 172.21.15.179:0/935519744 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9c30063c40 con 0x7f9c200b2900 2024-05-27T08:34:15.653 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mds.all update... 2024-05-27T08:34:15.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9c200b2900 msgr2=0x7f9c200b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:15.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9c200b2900 0x7f9c200b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c30102710 tx=0x7f9c1c005c10 comp rx=0 tx=0).stop 2024-05-27T08:34:15.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30102740 msgr2=0x7f9c30102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:15.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30102740 0x7f9c30102190 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9c240036f0 tx=0x7f9c240039f0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.655 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 shutdown_connections 2024-05-27T08:34:15.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c30103b70 0x7f9c301a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9c200b2900 0x7f9c200b4dc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30102740 0x7f9c30102190 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 --2- 172.21.15.179:0/935519744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c30103100 0x7f9c3019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:15.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 >> 172.21.15.179:0/935519744 conn(0x7f9c300fd210 msgr2=0x7f9c30106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:15.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 shutdown_connections 2024-05-27T08:34:15.656 INFO:teuthology.orchestra.run.smithi179.stderr:2024-05-27T08:34:15.654+0000 7f9c36efa700 1 -- 172.21.15.179:0/935519744 wait complete. 2024-05-27T08:34:16.346 DEBUG:teuthology.orchestra.run.smithi077:mds.a> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.a.service 2024-05-27T08:34:16.349 DEBUG:teuthology.orchestra.run.smithi077:mds.d> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.d.service 2024-05-27T08:34:16.393 DEBUG:teuthology.orchestra.run.smithi077:mds.g> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.g.service 2024-05-27T08:34:16.438 DEBUG:teuthology.orchestra.run.smithi077:mds.j> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.j.service 2024-05-27T08:34:16.481 DEBUG:teuthology.orchestra.run.smithi092:mds.b> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.b.service 2024-05-27T08:34:16.483 DEBUG:teuthology.orchestra.run.smithi092:mds.e> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.e.service 2024-05-27T08:34:16.526 DEBUG:teuthology.orchestra.run.smithi092:mds.h> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.h.service 2024-05-27T08:34:16.569 DEBUG:teuthology.orchestra.run.smithi092:mds.k> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.k.service 2024-05-27T08:34:16.612 DEBUG:teuthology.orchestra.run.smithi179:mds.c> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.c.service 2024-05-27T08:34:16.614 DEBUG:teuthology.orchestra.run.smithi179:mds.f> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.f.service 2024-05-27T08:34:16.657 DEBUG:teuthology.orchestra.run.smithi179:mds.i> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.i.service 2024-05-27T08:34:16.699 DEBUG:teuthology.orchestra.run.smithi179:mds.l> sudo journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.l.service 2024-05-27T08:34:16.741 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-27T08:34:16.741 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:34:16.977 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:16.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:16.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:34:16.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-05-27T08:34:16.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:16.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:16.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:16.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:16.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:16.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:16.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:16.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:16 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:17.099 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:17.100 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:16 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:17.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:17.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:34:17.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: Saving service mds.all spec with placement smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-05-27T08:34:17.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:17.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:17.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:17.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:17.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:17.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:17.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:17.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:16 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:17.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:17 smithi179 ceph-mon[36481]: Deploying daemon mds.l on smithi179 2024-05-27T08:34:17.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:17 smithi077 ceph-mon[32007]: Deploying daemon mds.l on smithi179 2024-05-27T08:34:17.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:17 smithi092 ceph-mon[36049]: Deploying daemon mds.l on smithi179 2024-05-27T08:34:18.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 -- 172.21.15.77:0/56602723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 msgr2=0x7f2d841088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:18.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 --2- 172.21.15.77:0/56602723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 0x7f2d841088f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009990 tx=0x7f2d74009ca0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 -- 172.21.15.77:0/56602723 shutdown_connections 2024-05-27T08:34:18.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 --2- 172.21.15.77:0/56602723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d84103f30 0x7f2d84110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 --2- 172.21.15.77:0/56602723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 0x7f2d841088f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 --2- 172.21.15.77:0/56602723 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d84103590 0x7f2d841039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 -- 172.21.15.77:0/56602723 >> 172.21.15.77:0/56602723 conn(0x7f2d8407a6f0 msgr2=0x7f2d8407ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:18.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 -- 172.21.15.77:0/56602723 shutdown_connections 2024-05-27T08:34:18.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 -- 172.21.15.77:0/56602723 wait complete. 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.541+0000 7f2d891a0700 1 Processor -- start 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d891a0700 1 -- start start 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d891a0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d84103590 0x7f2d8419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d891a0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d84103f30 0x7f2d8419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d891a0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 0x7f2d84199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d891a0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84113710 con 0x7f2d84108510 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d891a0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84113590 con 0x7f2d84103590 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d891a0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84113890 con 0x7f2d84103f30 2024-05-27T08:34:18.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 0x7f2d84199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 0x7f2d84199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44962/0 (socket says 172.21.15.77:44962) 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8359e700 1 -- 172.21.15.77:0/4233773907 learned_addr learned my addr 172.21.15.77:0/4233773907 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8259c700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d84103f30 0x7f2d8419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d82d9d700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d84103590 0x7f2d8419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8359e700 1 -- 172.21.15.77:0/4233773907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d84103f30 msgr2=0x7f2d8419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8359e700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d84103f30 0x7f2d8419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8359e700 1 -- 172.21.15.77:0/4233773907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d84103590 msgr2=0x7f2d8419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8359e700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d84103590 0x7f2d8419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.542+0000 7f2d8359e700 1 -- 172.21.15.77:0/4233773907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8419a1f0 con 0x7f2d84108510 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.543+0000 7f2d8359e700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 0x7f2d84199a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c007ee0 tx=0x7f2d7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.543+0000 7f2d6bfff700 1 -- 172.21.15.77:0/4233773907 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c004bb0 con 0x7f2d84108510 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.543+0000 7f2d6bfff700 1 -- 172.21.15.77:0/4233773907 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d7c004d50 con 0x7f2d84108510 2024-05-27T08:34:18.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.543+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8419a4e0 con 0x7f2d84108510 2024-05-27T08:34:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.543+0000 7f2d6bfff700 1 -- 172.21.15.77:0/4233773907 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c005920 con 0x7f2d84108510 2024-05-27T08:34:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.543+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d841a67a0 con 0x7f2d84108510 2024-05-27T08:34:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.544+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8406a230 con 0x7f2d84108510 2024-05-27T08:34:18.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.545+0000 7f2d6bfff700 1 -- 172.21.15.77:0/4233773907 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2d7c01a020 con 0x7f2d84108510 2024-05-27T08:34:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.546+0000 7f2d6bfff700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d6c0b2880 0x7f2d6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.546+0000 7f2d82d9d700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d6c0b2880 0x7f2d6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.546+0000 7f2d6bfff700 1 -- 172.21.15.77:0/4233773907 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f2d7c0e1a00 con 0x7f2d84108510 2024-05-27T08:34:18.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.546+0000 7f2d82d9d700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d6c0b2880 0x7f2d6c0b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006010 tx=0x7f2d74005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:18.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.549+0000 7f2d6bfff700 1 -- 172.21.15.77:0/4233773907 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d7c010070 con 0x7f2d84108510 2024-05-27T08:34:18.606 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 systemd[1]: Starting Ceph mds.l for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:34:18.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.808+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d8410bad0 con 0x7f2d84108510 2024-05-27T08:34:18.811 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.809+0000 7f2d6bfff700 1 -- 172.21.15.77:0/4233773907 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f2d7c0a9bc0 con 0x7f2d84108510 2024-05-27T08:34:18.811 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:18.811 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-27T08:34:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d6c0b2880 msgr2=0x7f2d6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:18.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d6c0b2880 0x7f2d6c0b4d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006010 tx=0x7f2d74005960 comp rx=0 tx=0).stop 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 msgr2=0x7f2d84199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 0x7f2d84199a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c007ee0 tx=0x7f2d7c00d3b0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 shutdown_connections 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d84108510 0x7f2d84199a90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d84103590 0x7f2d8419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d6c0b2880 0x7f2d6c0b4d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 --2- 172.21.15.77:0/4233773907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d84103f30 0x7f2d8419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 >> 172.21.15.77:0/4233773907 conn(0x7f2d8407a6f0 msgr2=0x7f2d84101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 shutdown_connections 2024-05-27T08:34:18.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:18.811+0000 7f2d891a0700 1 -- 172.21.15.77:0/4233773907 wait complete. 2024-05-27T08:34:18.815 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 1 2024-05-27T08:34:18.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:18 smithi179 ceph-mon[36481]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:18.922 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 podman[66714]: 2024-05-27 08:34:18.677179361 +0000 UTC m=+0.111487834 container create 4746f09de9fab0604147135662803d976a1ed124818409a9f28d990d18294b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-27T08:34:18.922 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 podman[66714]: 2024-05-27 08:34:18.58287123 +0000 UTC m=+0.017179696 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:34:18.922 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 podman[66714]: 2024-05-27 08:34:18.896108367 +0000 UTC m=+0.330416810 container init 4746f09de9fab0604147135662803d976a1ed124818409a9f28d990d18294b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2) 2024-05-27T08:34:18.922 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 podman[66714]: 2024-05-27 08:34:18.898585331 +0000 UTC m=+0.332893790 container start 4746f09de9fab0604147135662803d976a1ed124818409a9f28d990d18294b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-27T08:34:18.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:18 smithi092 ceph-mon[36049]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:19.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:18 smithi077 ceph-mon[32007]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:19.356 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 ceph-mds[66733]: -- 172.21.15.179:0/1283773927 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bb2c509a40 con 0x55bb2c4dc800 2024-05-27T08:34:19.356 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l[66729]: starting mds.l at 2024-05-27T08:34:19.356 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 bash[66714]: 4746f09de9fab0604147135662803d976a1ed124818409a9f28d990d18294b26 2024-05-27T08:34:19.357 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:34:18 smithi179 systemd[1]: Started Ceph mds.l for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/4233773907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6832/1008462834,v1:172.21.15.179:6833/1008462834] up:boot 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: fsmap 1 up:standby 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:19.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:19 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:19.911 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/4233773907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:19.911 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6832/1008462834,v1:172.21.15.179:6833/1008462834] up:boot 2024-05-27T08:34:19.911 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: fsmap 1 up:standby 2024-05-27T08:34:19.911 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:34:19.911 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:19.911 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:19.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:19.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:19.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:19.912 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:19 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:20.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/4233773907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6832/1008462834,v1:172.21.15.179:6833/1008462834] up:boot 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: fsmap 1 up:standby 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:20.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:19 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:20.964 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:20 smithi092 ceph-mon[36049]: Deploying daemon mds.k on smithi092 2024-05-27T08:34:20.964 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:20 smithi092 ceph-mon[36049]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:20 smithi179 ceph-mon[36481]: Deploying daemon mds.k on smithi092 2024-05-27T08:34:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:20 smithi179 ceph-mon[36481]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:21.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:20 smithi077 ceph-mon[32007]: Deploying daemon mds.k on smithi092 2024-05-27T08:34:21.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:20 smithi077 ceph-mon[32007]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:21.236 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 systemd[1]: Starting Ceph mds.k for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:34:21.624 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:34:21.657 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 podman[65703]: 2024-05-27 08:34:21.287661625 +0000 UTC m=+0.018118366 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:34:21.657 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 podman[65703]: 2024-05-27 08:34:21.393230208 +0000 UTC m=+0.123686949 container create d6c6c8fda2d85ed86043eb4ecd6e34218362c41715a11b0bbfccc2f19b2de91c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T08:34:21.657 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 podman[65703]: 2024-05-27 08:34:21.630072892 +0000 UTC m=+0.360529641 container init d6c6c8fda2d85ed86043eb4ecd6e34218362c41715a11b0bbfccc2f19b2de91c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:34:21.657 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 podman[65703]: 2024-05-27 08:34:21.632965834 +0000 UTC m=+0.363422576 container start d6c6c8fda2d85ed86043eb4ecd6e34218362c41715a11b0bbfccc2f19b2de91c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:34:21.877 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:21.986 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 ceph-mds[65722]: -- 172.21.15.92:0/1751691157 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643192b9a40 con 0x56431928cc00 2024-05-27T08:34:21.986 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k[65718]: starting mds.k at 2024-05-27T08:34:21.986 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 bash[65703]: d6c6c8fda2d85ed86043eb4ecd6e34218362c41715a11b0bbfccc2f19b2de91c 2024-05-27T08:34:21.986 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:34:21 smithi092 systemd[1]: Started Ceph mds.k for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:23.015 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: Deploying daemon mds.j on smithi077 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6832/1631050328,v1:172.21.15.92:6833/1631050328] up:boot 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: fsmap 2 up:standby 2024-05-27T08:34:23.016 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:22 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:34:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:23.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:23.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:23.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: Deploying daemon mds.j on smithi077 2024-05-27T08:34:23.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6832/1631050328,v1:172.21.15.92:6833/1631050328] up:boot 2024-05-27T08:34:23.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: fsmap 2 up:standby 2024-05-27T08:34:23.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:22 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:34:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: Deploying daemon mds.j on smithi077 2024-05-27T08:34:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6832/1631050328,v1:172.21.15.92:6833/1631050328] up:boot 2024-05-27T08:34:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: fsmap 2 up:standby 2024-05-27T08:34:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:22 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 -- 172.21.15.77:0/2218823657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 msgr2=0x7f05c0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/2218823657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 0x7f05c0108900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009990 tx=0x7f05b4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 -- 172.21.15.77:0/2218823657 shutdown_connections 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/2218823657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05c0102e00 0x7f05c010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/2218823657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 0x7f05c0108900 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/2218823657 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c0102440 0x7f05c01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 -- 172.21.15.77:0/2218823657 >> 172.21.15.77:0/2218823657 conn(0x7f05c00793f0 msgr2=0x7f05c0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 -- 172.21.15.77:0/2218823657 shutdown_connections 2024-05-27T08:34:23.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.613+0000 7f05c6cbf700 1 -- 172.21.15.77:0/2218823657 wait complete. 2024-05-27T08:34:23.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.614+0000 7f05c6cbf700 1 Processor -- start 2024-05-27T08:34:23.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.614+0000 7f05c6cbf700 1 -- start start 2024-05-27T08:34:23.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.614+0000 7f05c6cbf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c0102440 0x7f05c019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:23.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.614+0000 7f05c6cbf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05c0102e00 0x7f05c019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:23.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.614+0000 7f05c6cbf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 0x7f05c01a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:23.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.614+0000 7f05c6cbf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01125f0 con 0x7f05c0108520 2024-05-27T08:34:23.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.614+0000 7f05c6cbf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0112470 con 0x7f05c0102440 2024-05-27T08:34:23.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.614+0000 7f05c6cbf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0112770 con 0x7f05c0102e00 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 0x7f05c01a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c4a5b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c0102440 0x7f05c019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 0x7f05c01a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39916/0 (socket says 172.21.15.77:39916) 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 -- 172.21.15.77:0/3715654052 learned_addr learned my addr 172.21.15.77:0/3715654052 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 -- 172.21.15.77:0/3715654052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05c0102e00 msgr2=0x7f05c019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05bffff700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05c0102e00 0x7f05c019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05c0102e00 0x7f05c019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 -- 172.21.15.77:0/3715654052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c0102440 msgr2=0x7f05c019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c0102440 0x7f05c019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 -- 172.21.15.77:0/3715654052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c01a5ff0 con 0x7f05c0108520 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c525c700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 0x7f05c01a5ab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05b8007ee0 tx=0x7f05b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05bdffb700 1 -- 172.21.15.77:0/3715654052 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b800dbd0 con 0x7f05c0108520 2024-05-27T08:34:23.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c01a6220 con 0x7f05c0108520 2024-05-27T08:34:23.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05bdffb700 1 -- 172.21.15.77:0/3715654052 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b800dd70 con 0x7f05c0108520 2024-05-27T08:34:23.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05bdffb700 1 -- 172.21.15.77:0/3715654052 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8005990 con 0x7f05c0108520 2024-05-27T08:34:23.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.615+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c01a67b0 con 0x7f05c0108520 2024-05-27T08:34:23.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.616+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c010a9b0 con 0x7f05c0108520 2024-05-27T08:34:23.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.618+0000 7f05bdffb700 1 -- 172.21.15.77:0/3715654052 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f05b801a020 con 0x7f05c0108520 2024-05-27T08:34:23.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.618+0000 7f05bdffb700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f05b00b2880 0x7f05b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:23.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.618+0000 7f05c4a5b700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f05b00b2880 0x7f05b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:23.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.618+0000 7f05bdffb700 1 -- 172.21.15.77:0/3715654052 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f05b80e1480 con 0x7f05c0108520 2024-05-27T08:34:23.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.619+0000 7f05c4a5b700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f05b00b2880 0x7f05b00b4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05b40098e0 tx=0x7f05b4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:23.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.623+0000 7f05bdffb700 1 -- 172.21.15.77:0/3715654052 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05b8010070 con 0x7f05c0108520 2024-05-27T08:34:23.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.884+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05c006b830 con 0x7f05c0108520 2024-05-27T08:34:23.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.885+0000 7f05bdffb700 1 -- 172.21.15.77:0/3715654052 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f05b80a9640 con 0x7f05c0108520 2024-05-27T08:34:23.887 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:23.887 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T08:34:23.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f05b00b2880 msgr2=0x7f05b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:23.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f05b00b2880 0x7f05b00b4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f05b40098e0 tx=0x7f05b4005960 comp rx=0 tx=0).stop 2024-05-27T08:34:23.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 msgr2=0x7f05c01a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 0x7f05c01a5ab0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05b8007ee0 tx=0x7f05b800d3b0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 shutdown_connections 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108520 0x7f05c01a5ab0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f05c0102440 0x7f05c019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f05b00b2880 0x7f05b00b4d40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 --2- 172.21.15.77:0/3715654052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05c0102e00 0x7f05c019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 >> 172.21.15.77:0/3715654052 conn(0x7f05c00793f0 msgr2=0x7f05c0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 shutdown_connections 2024-05-27T08:34:23.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:23.887+0000 7f05c6cbf700 1 -- 172.21.15.77:0/3715654052 wait complete. 2024-05-27T08:34:23.891 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 3 2024-05-27T08:34:24.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:24 smithi077 ceph-mon[32007]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:24.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:24 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3715654052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:24 smithi179 ceph-mon[36481]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:24 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3715654052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:25.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:24 smithi092 ceph-mon[36049]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:25.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:24 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3715654052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:25.962 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:34:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-j[72305]: starting mds.j at 2024-05-27T08:34:26.379 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:34:25 smithi077 bash[72290]: 1d9ce26d82a5b9d4f199d8094ad25ffcd1070cf5faccb56a33f7d24b1102c88c 2024-05-27T08:34:26.380 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:34:25 smithi077 systemd[1]: Started Ceph mds.j for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:27.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:27.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:27.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.77:6834/3520949826,v1:172.21.15.77:6835/3520949826] up:boot 2024-05-27T08:34:27.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: fsmap 3 up:standby 2024-05-27T08:34:27.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:27 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T08:34:27.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:27.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:27.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:27.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:27.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.77:6834/3520949826,v1:172.21.15.77:6835/3520949826] up:boot 2024-05-27T08:34:27.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: fsmap 3 up:standby 2024-05-27T08:34:27.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:27 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T08:34:27.480 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:34:27.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:27.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:27.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:27.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:27.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:27.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.77:6834/3520949826,v1:172.21.15.77:6835/3520949826] up:boot 2024-05-27T08:34:27.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: fsmap 3 up:standby 2024-05-27T08:34:27.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:27 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T08:34:27.908 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:28.071 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:28 smithi077 ceph-mon[32007]: Deploying daemon mds.d on smithi077 2024-05-27T08:34:28.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:28 smithi179 ceph-mon[36481]: Deploying daemon mds.d on smithi077 2024-05-27T08:34:28.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:28 smithi092 ceph-mon[36049]: Deploying daemon mds.d on smithi077 2024-05-27T08:34:29.322 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:29 smithi077 ceph-mon[32007]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:29.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:29 smithi179 ceph-mon[36481]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:29.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:29 smithi092 ceph-mon[36049]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:29.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.699+0000 7f2f44af5700 1 -- 172.21.15.77:0/2399972021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 msgr2=0x7f2f401074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:29.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.699+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2399972021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 0x7f2f401074c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009990 tx=0x7f2f34009ca0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 -- 172.21.15.77:0/2399972021 shutdown_connections 2024-05-27T08:34:29.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2399972021 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f40107a00 0x7f2f4010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2399972021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f401029e0 0x7f2f40102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2399972021 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 0x7f2f401074c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 -- 172.21.15.77:0/2399972021 >> 172.21.15.77:0/2399972021 conn(0x7f2f40079330 msgr2=0x7f2f40079740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:29.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 -- 172.21.15.77:0/2399972021 shutdown_connections 2024-05-27T08:34:29.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 -- 172.21.15.77:0/2399972021 wait complete. 2024-05-27T08:34:29.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 Processor -- start 2024-05-27T08:34:29.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.700+0000 7f2f44af5700 1 -- start start 2024-05-27T08:34:29.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f44af5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f401029e0 0x7f2f40104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:29.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f44af5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 0x7f2f40104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:29.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f44af5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f40107a00 0x7f2f40104b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f44af5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40112660 con 0x7f2f40107060 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f44af5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2f401124e0 con 0x7f2f401029e0 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f44af5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2f401127e0 con 0x7f2f40107a00 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 0x7f2f40104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 0x7f2f40104640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33760/0 (socket says 172.21.15.77:33760) 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 -- 172.21.15.77:0/2355971874 learned_addr learned my addr 172.21.15.77:0/2355971874 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3e59c700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f401029e0 0x7f2f40104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3ed9d700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f40107a00 0x7f2f40104b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 -- 172.21.15.77:0/2355971874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f40107a00 msgr2=0x7f2f40104b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f40107a00 0x7f2f40104b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 -- 172.21.15.77:0/2355971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f401029e0 msgr2=0x7f2f40104100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f401029e0 0x7f2f40104100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 -- 172.21.15.77:0/2355971874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f401a81b0 con 0x7f2f40107060 2024-05-27T08:34:29.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f3dd9b700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 0x7f2f40104640 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f34000c00 tx=0x7f2f340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:29.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f2f7fe700 1 -- 172.21.15.77:0/2355971874 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34027070 con 0x7f2f40107060 2024-05-27T08:34:29.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.701+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f401a8440 con 0x7f2f40107060 2024-05-27T08:34:29.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.702+0000 7f2f2f7fe700 1 -- 172.21.15.77:0/2355971874 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f34003f20 con 0x7f2f40107060 2024-05-27T08:34:29.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.702+0000 7f2f2f7fe700 1 -- 172.21.15.77:0/2355971874 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34022ca0 con 0x7f2f40107060 2024-05-27T08:34:29.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.702+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f401a88f0 con 0x7f2f40107060 2024-05-27T08:34:29.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.702+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f40103bb0 con 0x7f2f40107060 2024-05-27T08:34:29.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.704+0000 7f2f2f7fe700 1 -- 172.21.15.77:0/2355971874 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f2f3402a070 con 0x7f2f40107060 2024-05-27T08:34:29.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.704+0000 7f2f2f7fe700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2f280b2880 0x7f2f280b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:29.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.705+0000 7f2f3e59c700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2f280b2880 0x7f2f280b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:29.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.705+0000 7f2f2f7fe700 1 -- 172.21.15.77:0/2355971874 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f2f340e3290 con 0x7f2f40107060 2024-05-27T08:34:29.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.705+0000 7f2f3e59c700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2f280b2880 0x7f2f280b4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009a10 tx=0x7f2f30009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:29.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.708+0000 7f2f2f7fe700 1 -- 172.21.15.77:0/2355971874 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f34031070 con 0x7f2f40107060 2024-05-27T08:34:29.880 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:34:29 smithi077 podman[72715]: 2024-05-27 08:34:29.840192847 +0000 UTC m=+0.448608843 container create 161d1c4d41cdaf583860bcd9ebed343c6e4953127f2187d5b12741a58caf7c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-27T08:34:29.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.967+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f401a8b50 con 0x7f2f40107060 2024-05-27T08:34:29.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.967+0000 7f2f2f7fe700 1 -- 172.21.15.77:0/2355971874 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f2f34013070 con 0x7f2f40107060 2024-05-27T08:34:29.969 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:29.969 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-27T08:34:29.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2f280b2880 msgr2=0x7f2f280b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:29.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2f280b2880 0x7f2f280b4d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009a10 tx=0x7f2f30009190 comp rx=0 tx=0).stop 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 msgr2=0x7f2f40104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 0x7f2f40104640 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f34000c00 tx=0x7f2f340047b0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 shutdown_connections 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2f280b2880 0x7f2f280b4d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f40107a00 0x7f2f40104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2f401029e0 0x7f2f40104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 --2- 172.21.15.77:0/2355971874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2f40107060 0x7f2f40104640 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 >> 172.21.15.77:0/2355971874 conn(0x7f2f40079330 msgr2=0x7f2f401063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 shutdown_connections 2024-05-27T08:34:29.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:29.969+0000 7f2f44af5700 1 -- 172.21.15.77:0/2355971874 wait complete. 2024-05-27T08:34:29.972 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 4 2024-05-27T08:34:30.278 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:30 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2355971874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:30.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:30 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2355971874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:30.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:30 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2355971874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:30.539 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:34:30 smithi077 podman[72715]: 2024-05-27 08:34:30.277346544 +0000 UTC m=+0.885762540 container init 161d1c4d41cdaf583860bcd9ebed343c6e4953127f2187d5b12741a58caf7c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:34:30.539 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:34:30 smithi077 podman[72715]: 2024-05-27 08:34:30.280525472 +0000 UTC m=+0.888941477 container start 161d1c4d41cdaf583860bcd9ebed343c6e4953127f2187d5b12741a58caf7c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:34:30.539 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:34:30 smithi077 ceph-mds[72769]: -- 172.21.15.77:0/2118027949 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b6040da40 con 0x561b603e0c00 2024-05-27T08:34:30.539 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:34:30 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d[72765]: starting mds.d at 2024-05-27T08:34:30.539 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:34:30 smithi077 bash[72715]: 161d1c4d41cdaf583860bcd9ebed343c6e4953127f2187d5b12741a58caf7c3a 2024-05-27T08:34:30.539 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:34:30 smithi077 systemd[1]: Started Ceph mds.d for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:31.324 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:31 smithi092 ceph-mon[36049]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:31.324 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.324 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.324 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.324 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:31.324 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:31.324 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:31 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:31 smithi179 ceph-mon[36481]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:31.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:31 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:31 smithi077 ceph-mon[32007]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:31.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:31 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:32.352 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:32 smithi092 ceph-mon[36049]: Deploying daemon mds.e on smithi092 2024-05-27T08:34:32.352 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:32 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.77:6836/619760507,v1:172.21.15.77:6837/619760507] up:boot 2024-05-27T08:34:32.352 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:32 smithi092 ceph-mon[36049]: fsmap 4 up:standby 2024-05-27T08:34:32.352 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:32 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T08:34:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:32 smithi179 ceph-mon[36481]: Deploying daemon mds.e on smithi092 2024-05-27T08:34:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:32 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.77:6836/619760507,v1:172.21.15.77:6837/619760507] up:boot 2024-05-27T08:34:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:32 smithi179 ceph-mon[36481]: fsmap 4 up:standby 2024-05-27T08:34:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:32 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T08:34:32.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:32 smithi077 ceph-mon[32007]: Deploying daemon mds.e on smithi092 2024-05-27T08:34:32.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:32 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.77:6836/619760507,v1:172.21.15.77:6837/619760507] up:boot 2024-05-27T08:34:32.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:32 smithi077 ceph-mon[32007]: fsmap 4 up:standby 2024-05-27T08:34:32.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:32 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T08:34:32.736 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:32 smithi092 systemd[1]: Starting Ceph mds.e for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:34:32.918 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:34:33.042 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:32 smithi092 podman[66016]: 2024-05-27 08:34:32.648758377 +0000 UTC m=+0.017894406 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:34:33.042 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:32 smithi092 podman[66016]: 2024-05-27 08:34:32.770608818 +0000 UTC m=+0.139744820 container create e417bc2498d9314d4cc0842e4d568513b49a09f75daa2880f56e5f0466b73a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0) 2024-05-27T08:34:33.042 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:32 smithi092 podman[66016]: 2024-05-27 08:34:32.964648161 +0000 UTC m=+0.333784162 container init e417bc2498d9314d4cc0842e4d568513b49a09f75daa2880f56e5f0466b73a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:34:33.042 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:32 smithi092 podman[66016]: 2024-05-27 08:34:32.96736938 +0000 UTC m=+0.336505395 container start e417bc2498d9314d4cc0842e4d568513b49a09f75daa2880f56e5f0466b73a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:34:33.042 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:32 smithi092 ceph-mds[66035]: -- 172.21.15.92:0/2347177553 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c917393a40 con 0x55c917366c00 2024-05-27T08:34:33.042 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:32 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e[66031]: starting mds.e at 2024-05-27T08:34:33.179 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:33.301 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:33 smithi077 ceph-mon[32007]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:33 smithi179 ceph-mon[36481]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:33.486 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:33 smithi092 bash[66016]: e417bc2498d9314d4cc0842e4d568513b49a09f75daa2880f56e5f0466b73a32 2024-05-27T08:34:33.486 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:34:33 smithi092 systemd[1]: Started Ceph mds.e for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:33.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:33 smithi092 ceph-mon[36049]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6834/4019780450,v1:172.21.15.92:6835/4019780450] up:boot 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: fsmap 5 up:standby 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: Deploying daemon mds.f on smithi179 2024-05-27T08:34:34.321 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:34 smithi077 ceph-mon[32007]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6834/4019780450,v1:172.21.15.92:6835/4019780450] up:boot 2024-05-27T08:34:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: fsmap 5 up:standby 2024-05-27T08:34:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:34:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:34.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:34.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:34.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: Deploying daemon mds.f on smithi179 2024-05-27T08:34:34.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:34 smithi179 ceph-mon[36481]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:34.442 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6834/4019780450,v1:172.21.15.92:6835/4019780450] up:boot 2024-05-27T08:34:34.442 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: fsmap 5 up:standby 2024-05-27T08:34:34.442 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:34:34.442 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.442 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.442 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:34.442 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:34.443 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:34.443 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:34.443 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: Deploying daemon mds.f on smithi179 2024-05-27T08:34:34.443 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:34 smithi092 ceph-mon[36049]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 -- 172.21.15.77:0/1775123776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c1028c0 msgr2=0x7f852c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 --2- 172.21.15.77:0/1775123776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c1028c0 0x7f852c110330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f852400b0a0 tx=0x7f852400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:34:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 -- 172.21.15.77:0/1775123776 shutdown_connections 2024-05-27T08:34:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 --2- 172.21.15.77:0/1775123776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c1028c0 0x7f852c110330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 --2- 172.21.15.77:0/1775123776 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852c108620 0x7f852c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 --2- 172.21.15.77:0/1775123776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c101f00 0x7f852c102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:34.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 -- 172.21.15.77:0/1775123776 >> 172.21.15.77:0/1775123776 conn(0x7f852c077780 msgr2=0x7f852c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 -- 172.21.15.77:0/1775123776 shutdown_connections 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.977+0000 7f85347f2700 1 -- 172.21.15.77:0/1775123776 wait complete. 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f85347f2700 1 Processor -- start 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f85347f2700 1 -- start start 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f85347f2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c101f00 0x7f852c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f85347f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852c1028c0 0x7f852c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f85347f2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c108620 0x7f852c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f85347f2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f852c113770 con 0x7f852c101f00 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f85347f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1135f0 con 0x7f852c1028c0 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f85347f2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f852c1138f0 con 0x7f852c108620 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c101f00 0x7f852c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c101f00 0x7f852c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33806/0 (socket says 172.21.15.77:33806) 2024-05-27T08:34:34.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 -- 172.21.15.77:0/310291566 learned_addr learned my addr 172.21.15.77:0/310291566 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f8531d8d700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852c1028c0 0x7f852c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f8532d8f700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c108620 0x7f852c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 -- 172.21.15.77:0/310291566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c108620 msgr2=0x7f852c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c108620 0x7f852c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 -- 172.21.15.77:0/310291566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852c1028c0 msgr2=0x7f852c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852c1028c0 0x7f852c19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 -- 172.21.15.77:0/310291566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852c19a150 con 0x7f852c101f00 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.978+0000 7f853258e700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c101f00 0x7f852c19f830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c970 tx=0x7f851c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.979+0000 7f85237fe700 1 -- 172.21.15.77:0/310291566 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c004220 con 0x7f852c101f00 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.979+0000 7f85237fe700 1 -- 172.21.15.77:0/310291566 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f851c0043c0 con 0x7f852c101f00 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.979+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852c19a440 con 0x7f852c101f00 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.979+0000 7f85237fe700 1 -- 172.21.15.77:0/310291566 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c003c20 con 0x7f852c101f00 2024-05-27T08:34:34.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.979+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852c1a6700 con 0x7f852c101f00 2024-05-27T08:34:34.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.980+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852c06a230 con 0x7f852c101f00 2024-05-27T08:34:34.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.981+0000 7f85237fe700 1 -- 172.21.15.77:0/310291566 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f851c0076d0 con 0x7f852c101f00 2024-05-27T08:34:34.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.982+0000 7f85237fe700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f85180b2900 0x7f85180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:34.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.982+0000 7f8531d8d700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f85180b2900 0x7f85180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:34.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.982+0000 7f85237fe700 1 -- 172.21.15.77:0/310291566 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f851c0e1bf0 con 0x7f852c101f00 2024-05-27T08:34:34.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.982+0000 7f8531d8d700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f85180b2900 0x7f85180b4dc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f852c077c80 tx=0x7f852800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:34.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:34.985+0000 7f85237fe700 1 -- 172.21.15.77:0/310291566 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f851c014030 con 0x7f852c101f00 2024-05-27T08:34:35.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.234+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f852c02e560 con 0x7f852c101f00 2024-05-27T08:34:35.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.234+0000 7f85237fe700 1 -- 172.21.15.77:0/310291566 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f851c0a9e30 con 0x7f852c101f00 2024-05-27T08:34:35.236 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:35.236 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-27T08:34:35.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f85180b2900 msgr2=0x7f85180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:35.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f85180b2900 0x7f85180b4dc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f852c077c80 tx=0x7f852800b3c0 comp rx=0 tx=0).stop 2024-05-27T08:34:35.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c101f00 msgr2=0x7f852c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c101f00 0x7f852c19f830 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c970 tx=0x7f851c00cc80 comp rx=0 tx=0).stop 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 shutdown_connections 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f85180b2900 0x7f85180b4dc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f852c108620 0x7f852c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f852c101f00 0x7f852c19f830 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 --2- 172.21.15.77:0/310291566 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f852c1028c0 0x7f852c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 >> 172.21.15.77:0/310291566 conn(0x7f852c077780 msgr2=0x7f852c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 shutdown_connections 2024-05-27T08:34:35.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:35.237+0000 7f85347f2700 1 -- 172.21.15.77:0/310291566 wait complete. 2024-05-27T08:34:35.240 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 6 2024-05-27T08:34:35.356 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 systemd[1]: Starting Ceph mds.f for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:34:35.650 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 podman[67017]: 2024-05-27 08:34:35.395498795 +0000 UTC m=+0.114128922 container create 0b34f7b54725ae2af132814851ebe198a85705ce7f84d68edbdbde0026a5c518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-27T08:34:35.650 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 podman[67017]: 2024-05-27 08:34:35.298777975 +0000 UTC m=+0.017408118 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:34:35.650 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 podman[67017]: 2024-05-27 08:34:35.623186625 +0000 UTC m=+0.341816765 container init 0b34f7b54725ae2af132814851ebe198a85705ce7f84d68edbdbde0026a5c518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f, ceph=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T08:34:35.651 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 podman[67017]: 2024-05-27 08:34:35.625820519 +0000 UTC m=+0.344450647 container start 0b34f7b54725ae2af132814851ebe198a85705ce7f84d68edbdbde0026a5c518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-27T08:34:35.651 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 ceph-mds[67036]: -- 172.21.15.179:0/2232441797 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56121b261a40 con 0x56121b234800 2024-05-27T08:34:36.106 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f[67032]: starting mds.f at 2024-05-27T08:34:36.106 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 bash[67017]: 0b34f7b54725ae2af132814851ebe198a85705ce7f84d68edbdbde0026a5c518 2024-05-27T08:34:36.106 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:34:35 smithi179 systemd[1]: Started Ceph mds.f for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/310291566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:36.479 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:36.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:36.480 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:36 smithi179 ceph-mon[36481]: Deploying daemon mds.c on smithi179 2024-05-27T08:34:36.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:36.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/310291566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:36.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:36.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:36.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:36.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:36 smithi092 ceph-mon[36049]: Deploying daemon mds.c on smithi179 2024-05-27T08:34:36.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:36.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/310291566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:36.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:36.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:36.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:36.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:36.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:36 smithi077 ceph-mon[32007]: Deploying daemon mds.c on smithi179 2024-05-27T08:34:37.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:37 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] up:boot 2024-05-27T08:34:37.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:37 smithi179 ceph-mon[36481]: fsmap 6 up:standby 2024-05-27T08:34:37.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:37 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:34:37.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:37 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] up:boot 2024-05-27T08:34:37.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:37 smithi092 ceph-mon[36049]: fsmap 6 up:standby 2024-05-27T08:34:37.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:37 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:34:37.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:37 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] up:boot 2024-05-27T08:34:37.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:37 smithi077 ceph-mon[32007]: fsmap 6 up:standby 2024-05-27T08:34:37.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:37 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:34:37.818 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:37 smithi179 systemd[1]: Starting Ceph mds.c for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:34:37.818 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:37 smithi179 podman[67316]: 2024-05-27 08:34:37.719474189 +0000 UTC m=+0.017391889 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:34:37.868 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:34:38.068 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:37 smithi179 podman[67316]: 2024-05-27 08:34:37.822835314 +0000 UTC m=+0.120753084 container create de3049cdd64342f58f8db1d14f8bdbbfce2c7af9a6e6eb7bea88faf0dea9991b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux ) 2024-05-27T08:34:38.068 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:38 smithi179 podman[67316]: 2024-05-27 08:34:38.041907564 +0000 UTC m=+0.339825255 container init de3049cdd64342f58f8db1d14f8bdbbfce2c7af9a6e6eb7bea88faf0dea9991b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:34:38.068 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:38 smithi179 podman[67316]: 2024-05-27 08:34:38.044385381 +0000 UTC m=+0.342303074 container start de3049cdd64342f58f8db1d14f8bdbbfce2c7af9a6e6eb7bea88faf0dea9991b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T08:34:38.121 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-mon[36481]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:38.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-mds[67335]: -- 172.21.15.179:0/3093858757 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ccf9c5da40 con 0x55ccf9c31000 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:38 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c[67331]: starting mds.c at 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:38 smithi179 bash[67316]: de3049cdd64342f58f8db1d14f8bdbbfce2c7af9a6e6eb7bea88faf0dea9991b 2024-05-27T08:34:38.357 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:34:38 smithi179 systemd[1]: Started Ceph mds.c for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:38.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:38 smithi092 ceph-mon[36049]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:38.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:38.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:38.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:38 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:38.492 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:38 smithi077 ceph-mon[32007]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:38.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:38.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:38.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:38.493 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:38 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:39.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:39 smithi077 ceph-mon[32007]: Deploying daemon mds.h on smithi092 2024-05-27T08:34:39.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:39 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] up:boot 2024-05-27T08:34:39.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:39 smithi077 ceph-mon[32007]: fsmap 7 up:standby 2024-05-27T08:34:39.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:39 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:34:39.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:39 smithi179 ceph-mon[36481]: Deploying daemon mds.h on smithi092 2024-05-27T08:34:39.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:39 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] up:boot 2024-05-27T08:34:39.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:39 smithi179 ceph-mon[36481]: fsmap 7 up:standby 2024-05-27T08:34:39.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:39 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:34:39.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:39 smithi092 ceph-mon[36049]: Deploying daemon mds.h on smithi092 2024-05-27T08:34:39.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:39 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] up:boot 2024-05-27T08:34:39.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:39 smithi092 ceph-mon[36049]: fsmap 7 up:standby 2024-05-27T08:34:39.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:39 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 -- 172.21.15.77:0/780329874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0104200 msgr2=0x7f12b01081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 --2- 172.21.15.77:0/780329874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0104200 0x7f12b01081f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12a00099c0 tx=0x7f12a0009cd0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 -- 172.21.15.77:0/780329874 shutdown_connections 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 --2- 172.21.15.77:0/780329874 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12b0108920 0x7f12b01104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 --2- 172.21.15.77:0/780329874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b0103850 0x7f12b0103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 --2- 172.21.15.77:0/780329874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0104200 0x7f12b01081f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 -- 172.21.15.77:0/780329874 >> 172.21.15.77:0/780329874 conn(0x7f12b007a5d0 msgr2=0x7f12b007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 -- 172.21.15.77:0/780329874 shutdown_connections 2024-05-27T08:34:39.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.641+0000 7f12b4a22700 1 -- 172.21.15.77:0/780329874 wait complete. 2024-05-27T08:34:39.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12b4a22700 1 Processor -- start 2024-05-27T08:34:39.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12b4a22700 1 -- start start 2024-05-27T08:34:39.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12b4a22700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0103850 0x7f12b0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:39.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12b4a22700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b0104200 0x7f12b0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12b4a22700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12b0108920 0x7f12b00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12b4a22700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01136e0 con 0x7f12b0103850 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12b4a22700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0113560 con 0x7f12b0108920 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12b4a22700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0113860 con 0x7f12b0104200 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12ae59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0103850 0x7f12b0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12ae59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0103850 0x7f12b0075840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53642/0 (socket says 172.21.15.77:53642) 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.642+0000 7f12ae59c700 1 -- 172.21.15.77:0/609252225 learned_addr learned my addr 172.21.15.77:0/609252225 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12add9b700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b0104200 0x7f12b0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12aed9d700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12b0108920 0x7f12b00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12ae59c700 1 -- 172.21.15.77:0/609252225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b0104200 msgr2=0x7f12b0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12ae59c700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b0104200 0x7f12b0075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12ae59c700 1 -- 172.21.15.77:0/609252225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12b0108920 msgr2=0x7f12b00762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12ae59c700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12b0108920 0x7f12b00762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12ae59c700 1 -- 172.21.15.77:0/609252225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b01a80a0 con 0x7f12b0103850 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12ae59c700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0103850 0x7f12b0075840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12a400d950 tx=0x7f12a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f129f7fe700 1 -- 172.21.15.77:0/609252225 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4016070 con 0x7f12b0103850 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b01a8390 con 0x7f12b0103850 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f129f7fe700 1 -- 172.21.15.77:0/609252225 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a400fd40 con 0x7f12b0103850 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f129f7fe700 1 -- 172.21.15.77:0/609252225 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a400e3b0 con 0x7f12b0103850 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.643+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01a88a0 con 0x7f12b0103850 2024-05-27T08:34:39.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.644+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b006a230 con 0x7f12b0103850 2024-05-27T08:34:39.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.645+0000 7f129f7fe700 1 -- 172.21.15.77:0/609252225 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f12a4019070 con 0x7f12b0103850 2024-05-27T08:34:39.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.646+0000 7f129f7fe700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f12980b2880 0x7f12980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:39.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.646+0000 7f12add9b700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f12980b2880 0x7f12980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:39.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.646+0000 7f129f7fe700 1 -- 172.21.15.77:0/609252225 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f12a40e1860 con 0x7f12b0103850 2024-05-27T08:34:39.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.646+0000 7f12add9b700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f12980b2880 0x7f12980b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009990 tx=0x7f12a0005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:39.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.651+0000 7f129f7fe700 1 -- 172.21.15.77:0/609252225 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f12a40a9a20 con 0x7f12b0103850 2024-05-27T08:34:39.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.900+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12b002e720 con 0x7f12b0103850 2024-05-27T08:34:39.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.901+0000 7f129f7fe700 1 -- 172.21.15.77:0/609252225 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f12a40a9320 con 0x7f12b0103850 2024-05-27T08:34:39.902 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:39.903 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-27T08:34:39.904 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f12980b2880 msgr2=0x7f12980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f12980b2880 0x7f12980b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009990 tx=0x7f12a0005c40 comp rx=0 tx=0).stop 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0103850 msgr2=0x7f12b0075840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0103850 0x7f12b0075840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12a400d950 tx=0x7f12a400dc60 comp rx=0 tx=0).stop 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 shutdown_connections 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12b0108920 0x7f12b00762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12b0103850 0x7f12b0075840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f12980b2880 0x7f12980b4d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 --2- 172.21.15.77:0/609252225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b0104200 0x7f12b0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 >> 172.21.15.77:0/609252225 conn(0x7f12b007a5d0 msgr2=0x7f12b01018e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 shutdown_connections 2024-05-27T08:34:39.905 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:39.903+0000 7f12b4a22700 1 -- 172.21.15.77:0/609252225 wait complete. 2024-05-27T08:34:39.906 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 8 2024-05-27T08:34:40.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:40 smithi092 ceph-mon[36049]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:40.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:40 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/609252225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:40.487 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 systemd[1]: Starting Ceph mds.h for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:34:40.569 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:40 smithi077 ceph-mon[32007]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:40.570 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:40 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/609252225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:40 smithi179 ceph-mon[36481]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:40 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/609252225' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:40.847 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 podman[66316]: 2024-05-27 08:34:40.427798614 +0000 UTC m=+0.017125390 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:34:40.847 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 podman[66316]: 2024-05-27 08:34:40.533578622 +0000 UTC m=+0.122905378 container create 2ec7d89d40049ed2a796cb95c5ee11b591ac69a706c2d56e3953498790d17698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T08:34:40.847 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 podman[66316]: 2024-05-27 08:34:40.745492902 +0000 UTC m=+0.334819669 container init 2ec7d89d40049ed2a796cb95c5ee11b591ac69a706c2d56e3953498790d17698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T08:34:40.847 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 podman[66316]: 2024-05-27 08:34:40.748046712 +0000 UTC m=+0.337373470 container start 2ec7d89d40049ed2a796cb95c5ee11b591ac69a706c2d56e3953498790d17698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True) 2024-05-27T08:34:40.847 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 ceph-mds[66336]: -- 172.21.15.92:0/2984582218 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56071de53a40 con 0x56071de27000 2024-05-27T08:34:40.847 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h[66332]: starting mds.h at 2024-05-27T08:34:40.847 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 bash[66316]: 2ec7d89d40049ed2a796cb95c5ee11b591ac69a706c2d56e3953498790d17698 2024-05-27T08:34:41.236 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:34:40 smithi092 systemd[1]: Started Ceph mds.h for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:42.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: Deploying daemon mds.i on smithi179 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] up:boot 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: fsmap 8 up:standby 2024-05-27T08:34:42.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:41 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:34:42.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:42.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:42.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:42.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: Deploying daemon mds.i on smithi179 2024-05-27T08:34:42.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] up:boot 2024-05-27T08:34:42.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: fsmap 8 up:standby 2024-05-27T08:34:42.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:41 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:34:42.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:42.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:42.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:42.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:42.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: Deploying daemon mds.i on smithi179 2024-05-27T08:34:42.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] up:boot 2024-05-27T08:34:42.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: fsmap 8 up:standby 2024-05-27T08:34:42.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:41 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:34:42.598 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:34:42.851 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:43.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:42 smithi179 ceph-mon[36481]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:43.107 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:42 smithi179 systemd[1]: Starting Ceph mds.i for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:34:43.169 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:42 smithi092 ceph-mon[36049]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:43.174 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:42 smithi077 ceph-mon[32007]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:43.413 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:43 smithi179 podman[67630]: 2024-05-27 08:34:43.159395854 +0000 UTC m=+0.113759658 container create b660e7f6b09bead3af680a0df2be69071bdea27ce9b911fabdd2a0887d482f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5) 2024-05-27T08:34:43.413 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:43 smithi179 podman[67630]: 2024-05-27 08:34:43.063079504 +0000 UTC m=+0.017443298 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:34:43.413 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:43 smithi179 podman[67630]: 2024-05-27 08:34:43.386924295 +0000 UTC m=+0.341288077 container init b660e7f6b09bead3af680a0df2be69071bdea27ce9b911fabdd2a0887d482f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T08:34:43.413 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:43 smithi179 podman[67630]: 2024-05-27 08:34:43.389468273 +0000 UTC m=+0.343832062 container start b660e7f6b09bead3af680a0df2be69071bdea27ce9b911fabdd2a0887d482f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521) 2024-05-27T08:34:43.856 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:43 smithi179 ceph-mds[67649]: -- 172.21.15.179:0/2088845460 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597101bfa40 con 0x559710192800 2024-05-27T08:34:43.856 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:43 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i[67645]: starting mds.i at 2024-05-27T08:34:43.856 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:43 smithi179 bash[67630]: b660e7f6b09bead3af680a0df2be69071bdea27ce9b911fabdd2a0887d482f97 2024-05-27T08:34:43.856 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:34:43 smithi179 systemd[1]: Started Ceph mds.i for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:44.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 -- 172.21.15.77:0/2869178466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102e30 msgr2=0x7f936410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:44.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 --2- 172.21.15.77:0/2869178466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102e30 0x7f936410f2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f935c00b0a0 tx=0x7f935c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 -- 172.21.15.77:0/2869178466 shutdown_connections 2024-05-27T08:34:44.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 --2- 172.21.15.77:0/2869178466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102e30 0x7f936410f2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 --2- 172.21.15.77:0/2869178466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93641084b0 0x7f9364108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 --2- 172.21.15.77:0/2869178466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9364102470 0x7f93641028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 -- 172.21.15.77:0/2869178466 >> 172.21.15.77:0/2869178466 conn(0x7f9364079410 msgr2=0x7f9364079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 -- 172.21.15.77:0/2869178466 shutdown_connections 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 -- 172.21.15.77:0/2869178466 wait complete. 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.441+0000 7f9369e12700 1 Processor -- start 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9369e12700 1 -- start start 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9369e12700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102470 0x7f9364104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9369e12700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9364102e30 0x7f93641045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9369e12700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93641084b0 0x7f9364104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9369e12700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9364112640 con 0x7f9364102470 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9369e12700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f93641124c0 con 0x7f93641084b0 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9369e12700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93641127c0 con 0x7f9364102e30 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102470 0x7f9364104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102470 0x7f9364104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53668/0 (socket says 172.21.15.77:53668) 2024-05-27T08:34:44.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 -- 172.21.15.77:0/1987115256 learned_addr learned my addr 172.21.15.77:0/1987115256 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9362ffd700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9364102e30 0x7f93641045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9363fff700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93641084b0 0x7f9364104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 -- 172.21.15.77:0/1987115256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9364102e30 msgr2=0x7f93641045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9364102e30 0x7f93641045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 -- 172.21.15.77:0/1987115256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93641084b0 msgr2=0x7f9364104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93641084b0 0x7f9364104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 -- 172.21.15.77:0/1987115256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93641a8190 con 0x7f9364102470 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f9363fff700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93641084b0 0x7f9364104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.442+0000 7f93637fe700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102470 0x7f9364104080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f935400d950 tx=0x7f935400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:44.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.443+0000 7f9360ff9700 1 -- 172.21.15.77:0/1987115256 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354016070 con 0x7f9364102470 2024-05-27T08:34:44.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.443+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93641a8480 con 0x7f9364102470 2024-05-27T08:34:44.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.443+0000 7f9360ff9700 1 -- 172.21.15.77:0/1987115256 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9354010460 con 0x7f9364102470 2024-05-27T08:34:44.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.443+0000 7f9360ff9700 1 -- 172.21.15.77:0/1987115256 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935400f620 con 0x7f9364102470 2024-05-27T08:34:44.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.443+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93641a8a10 con 0x7f9364102470 2024-05-27T08:34:44.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.444+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936406a230 con 0x7f9364102470 2024-05-27T08:34:44.447 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.445+0000 7f9360ff9700 1 -- 172.21.15.77:0/1987115256 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9354019070 con 0x7f9364102470 2024-05-27T08:34:44.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.445+0000 7f9360ff9700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f934c0b2880 0x7f934c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:44.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.446+0000 7f9362ffd700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f934c0b2880 0x7f934c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:44.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.446+0000 7f9360ff9700 1 -- 172.21.15.77:0/1987115256 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f93540e1d90 con 0x7f9364102470 2024-05-27T08:34:44.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.446+0000 7f9362ffd700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f934c0b2880 0x7f934c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9358006fd0 tx=0x7f9358008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:44.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.451+0000 7f9360ff9700 1 -- 172.21.15.77:0/1987115256 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93540a9f50 con 0x7f9364102470 2024-05-27T08:34:44.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.717+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93641055b0 con 0x7f9364102470 2024-05-27T08:34:44.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.717+0000 7f9360ff9700 1 -- 172.21.15.77:0/1987115256 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f93540a9850 con 0x7f9364102470 2024-05-27T08:34:44.719 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:44.719 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24410,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24412,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-27T08:34:44.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.719+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f934c0b2880 msgr2=0x7f934c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:44.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.719+0000 7f9369e12700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f934c0b2880 0x7f934c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9358006fd0 tx=0x7f9358008040 comp rx=0 tx=0).stop 2024-05-27T08:34:44.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.719+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102470 msgr2=0x7f9364104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:44.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.719+0000 7f9369e12700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102470 0x7f9364104080 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f935400d950 tx=0x7f935400dd10 comp rx=0 tx=0).stop 2024-05-27T08:34:44.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.720+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 shutdown_connections 2024-05-27T08:34:44.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.720+0000 7f9369e12700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93641084b0 0x7f9364104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.720+0000 7f9369e12700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9364102470 0x7f9364104080 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.720+0000 7f9369e12700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f934c0b2880 0x7f934c0b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.720+0000 7f9369e12700 1 --2- 172.21.15.77:0/1987115256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9364102e30 0x7f93641045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:44.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.720+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 >> 172.21.15.77:0/1987115256 conn(0x7f9364079410 msgr2=0x7f936406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:44.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.720+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 shutdown_connections 2024-05-27T08:34:44.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:44.720+0000 7f9369e12700 1 -- 172.21.15.77:0/1987115256 wait complete. 2024-05-27T08:34:44.722 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 10 2024-05-27T08:34:44.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: Deploying daemon mds.g on smithi077 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] up:boot 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: fsmap 9 up:standby 2024-05-27T08:34:44.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:44 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:34:44.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: Deploying daemon mds.g on smithi077 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] up:boot 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: fsmap 9 up:standby 2024-05-27T08:34:44.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:44 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:34:44.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: Deploying daemon mds.g on smithi077 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] up:boot 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: fsmap 9 up:standby 2024-05-27T08:34:44.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:44 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:34:45.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:45 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1987115256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:45.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:45 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1987115256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:45.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:45 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1987115256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:46.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:46 smithi077 ceph-mon[32007]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:46 smithi179 ceph-mon[36481]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:46.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:46 smithi092 ceph-mon[36049]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:47.003 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:34:46 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g[73556]: starting mds.g at 2024-05-27T08:34:47.379 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:34:47 smithi077 bash[73540]: 37f65f013fbc28c55398956fbc7e4b040771810c103ee7f68b8bc974978a9745 2024-05-27T08:34:47.380 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:34:47 smithi077 systemd[1]: Started Ceph mds.g for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:47.610 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:34:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: Deploying daemon mds.a on smithi077 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.77:6838/3869213740,v1:172.21.15.77:6839/3869213740] up:boot 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: fsmap 10 up:standby 2024-05-27T08:34:48.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:34:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:48.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: Deploying daemon mds.a on smithi077 2024-05-27T08:34:48.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:48.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.77:6838/3869213740,v1:172.21.15.77:6839/3869213740] up:boot 2024-05-27T08:34:48.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: fsmap 10 up:standby 2024-05-27T08:34:48.381 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:34:48.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:48.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:48.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:48.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:48.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: Deploying daemon mds.a on smithi077 2024-05-27T08:34:48.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:48.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.77:6838/3869213740,v1:172.21.15.77:6839/3869213740] up:boot 2024-05-27T08:34:48.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: fsmap 10 up:standby 2024-05-27T08:34:48.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:34:48.774 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:50.310 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:50 smithi077 ceph-mon[32007]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:50.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:50 smithi092 ceph-mon[36049]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:50.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 -- 172.21.15.77:0/2451152468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f01035d0 msgr2=0x7f28f0103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 --2- 172.21.15.77:0/2451152468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f01035d0 0x7f28f0103a50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f28d80099c0 tx=0x7f28d8009cd0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 -- 172.21.15.77:0/2451152468 shutdown_connections 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 --2- 172.21.15.77:0/2451152468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28f0103f90 0x7f28f0110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 --2- 172.21.15.77:0/2451152468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28f0109670 0x7f28f0109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 --2- 172.21.15.77:0/2451152468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f01035d0 0x7f28f0103a50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 -- 172.21.15.77:0/2451152468 >> 172.21.15.77:0/2451152468 conn(0x7f28f007a730 msgr2=0x7f28f007ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 -- 172.21.15.77:0/2451152468 shutdown_connections 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.542+0000 7f28f5a89700 1 -- 172.21.15.77:0/2451152468 wait complete. 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28f5a89700 1 Processor -- start 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28f5a89700 1 -- start start 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28f5a89700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28f01035d0 0x7f28f019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28f5a89700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28f0103f90 0x7f28f019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:50.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28f5a89700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f0109670 0x7f28f0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28f5a89700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28f01137d0 con 0x7f28f0109670 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28f5a89700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f28f0113650 con 0x7f28f01035d0 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28f5a89700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28f0113950 con 0x7f28f0103f90 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28effff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f0109670 0x7f28f0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28effff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f0109670 0x7f28f0199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40748/0 (socket says 172.21.15.77:40748) 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28effff700 1 -- 172.21.15.77:0/1618808311 learned_addr learned my addr 172.21.15.77:0/1618808311 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.543+0000 7f28e7fff700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28f0103f90 0x7f28f019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28ef7fe700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28f01035d0 0x7f28f019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28effff700 1 -- 172.21.15.77:0/1618808311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28f0103f90 msgr2=0x7f28f019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28effff700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28f0103f90 0x7f28f019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28effff700 1 -- 172.21.15.77:0/1618808311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28f01035d0 msgr2=0x7f28f019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28effff700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28f01035d0 0x7f28f019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28effff700 1 -- 172.21.15.77:0/1618808311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f019a1f0 con 0x7f28f0109670 2024-05-27T08:34:50.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28effff700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f0109670 0x7f28f0199a90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b970 tx=0x7f28e800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28ed7fa700 1 -- 172.21.15.77:0/1618808311 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e8019070 con 0x7f28f0109670 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28ed7fa700 1 -- 172.21.15.77:0/1618808311 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e8014440 con 0x7f28f0109670 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f019a4e0 con 0x7f28f0109670 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28ed7fa700 1 -- 172.21.15.77:0/1618808311 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e800ce60 con 0x7f28f0109670 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.544+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f01a67a0 con 0x7f28f0109670 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.545+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f006a230 con 0x7f28f0109670 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.546+0000 7f28ed7fa700 1 -- 172.21.15.77:0/1618808311 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f28e801c070 con 0x7f28f0109670 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.547+0000 7f28ed7fa700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28dc0b2880 0x7f28dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.547+0000 7f28ef7fe700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28dc0b2880 0x7f28dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.547+0000 7f28ed7fa700 1 -- 172.21.15.77:0/1618808311 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f28e80e1180 con 0x7f28f0109670 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.548+0000 7f28ef7fe700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28dc0b2880 0x7f28dc0b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f28f019abd0 tx=0x7f28e000b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:50.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.552+0000 7f28ed7fa700 1 -- 172.21.15.77:0/1618808311 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28e80a9340 con 0x7f28f0109670 2024-05-27T08:34:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:50 smithi179 ceph-mon[36481]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:50.814 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.813+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28f019afc0 con 0x7f28f0109670 2024-05-27T08:34:50.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.814+0000 7f28ed7fa700 1 -- 172.21.15.77:0/1618808311 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f28e80a8c40 con 0x7f28f0109670 2024-05-27T08:34:50.815 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:50.815 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24410,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24412,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28dc0b2880 msgr2=0x7f28dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28dc0b2880 0x7f28dc0b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f28f019abd0 tx=0x7f28e000b5d0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f0109670 msgr2=0x7f28f0199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f0109670 0x7f28f0199a90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b970 tx=0x7f28e800bd30 comp rx=0 tx=0).stop 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 shutdown_connections 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28f0109670 0x7f28f0199a90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28f01035d0 0x7f28f019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f28dc0b2880 0x7f28dc0b4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 --2- 172.21.15.77:0/1618808311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28f0103f90 0x7f28f019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:50.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 >> 172.21.15.77:0/1618808311 conn(0x7f28f007a730 msgr2=0x7f28f0102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 shutdown_connections 2024-05-27T08:34:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:50.816+0000 7f28f5a89700 1 -- 172.21.15.77:0/1618808311 wait complete. 2024-05-27T08:34:50.819 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 11 2024-05-27T08:34:50.943 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:34:50 smithi077 podman[73962]: 2024-05-27 08:34:50.918977259 +0000 UTC m=+0.573203323 container start f89bb5c0f0093a6550ed39a8d5fccc4f012e055bbeaf530b7f81d7c0d5f3c537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:34:50.944 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:34:50 smithi077 ceph-mds[74016]: -- 172.21.15.77:0/3744445900 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55891ca59a40 con 0x55891ca2d000 2024-05-27T08:34:51.199 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:34:50 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a[73999]: starting mds.a at 2024-05-27T08:34:51.199 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:34:51 smithi077 bash[73962]: f89bb5c0f0093a6550ed39a8d5fccc4f012e055bbeaf530b7f81d7c0d5f3c537 2024-05-27T08:34:51.199 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:34:51 smithi077 systemd[1]: Started Ceph mds.a for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:51 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1618808311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:51.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:51.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:51 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:51 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1618808311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:51.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:51.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:51 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:51 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1618808311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:51.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-27T08:34:51.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-27T08:34:51.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:51 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:52.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:52 smithi092 ceph-mon[36049]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:52.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:52 smithi092 ceph-mon[36049]: Deploying daemon mds.b on smithi092 2024-05-27T08:34:52.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:52 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.77:6840/906525687,v1:172.21.15.77:6841/906525687] up:boot 2024-05-27T08:34:52.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:52 smithi092 ceph-mon[36049]: fsmap 11 up:standby 2024-05-27T08:34:52.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:52 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:34:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:52 smithi179 ceph-mon[36481]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:52 smithi179 ceph-mon[36481]: Deploying daemon mds.b on smithi092 2024-05-27T08:34:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:52 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.77:6840/906525687,v1:172.21.15.77:6841/906525687] up:boot 2024-05-27T08:34:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:52 smithi179 ceph-mon[36481]: fsmap 11 up:standby 2024-05-27T08:34:52.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:52 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:34:52.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:52 smithi077 ceph-mon[32007]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:52.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:52 smithi077 ceph-mon[32007]: Deploying daemon mds.b on smithi092 2024-05-27T08:34:52.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:52 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.77:6840/906525687,v1:172.21.15.77:6841/906525687] up:boot 2024-05-27T08:34:52.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:52 smithi077 ceph-mon[32007]: fsmap 11 up:standby 2024-05-27T08:34:52.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:52 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:34:53.236 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 systemd[1]: Starting Ceph mds.b for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:34:53.653 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 podman[66633]: 2024-05-27 08:34:53.286750974 +0000 UTC m=+0.016855998 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:34:53.653 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 podman[66633]: 2024-05-27 08:34:53.390634001 +0000 UTC m=+0.120739008 container create e737fd95726e46f82767804184308de183c0fde7882079cb4d9af85c1e8c4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-27T08:34:53.653 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 podman[66633]: 2024-05-27 08:34:53.626566902 +0000 UTC m=+0.356671906 container init e737fd95726e46f82767804184308de183c0fde7882079cb4d9af85c1e8c4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-27T08:34:53.653 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 podman[66633]: 2024-05-27 08:34:53.629365113 +0000 UTC m=+0.359470129 container start e737fd95726e46f82767804184308de183c0fde7882079cb4d9af85c1e8c4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-27T08:34:53.653 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 ceph-mds[66653]: -- 172.21.15.92:0/3638747188 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55637c543a40 con 0x55637c517000 2024-05-27T08:34:53.667 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:34:53.925 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:53.986 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: starting mds.b at 2024-05-27T08:34:53.986 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 bash[66633]: e737fd95726e46f82767804184308de183c0fde7882079cb4d9af85c1e8c4f04 2024-05-27T08:34:53.986 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:34:53 smithi092 systemd[1]: Started Ceph mds.b for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] up:boot 2024-05-27T08:34:55.042 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: fsmap 12 up:standby 2024-05-27T08:34:55.043 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:34:55.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:55.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] up:boot 2024-05-27T08:34:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: fsmap 12 up:standby 2024-05-27T08:34:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:34:55.174 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:55.174 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.174 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.174 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.174 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.174 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:55.175 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:55.175 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] up:boot 2024-05-27T08:34:55.175 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: fsmap 12 up:standby 2024-05-27T08:34:55.175 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:34:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 -- 172.21.15.77:0/3788170470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51681095b0 msgr2=0x7f5168109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 --2- 172.21.15.77:0/3788170470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51681095b0 0x7f5168109990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5164009990 tx=0x7f5164009ca0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 -- 172.21.15.77:0/3788170470 shutdown_connections 2024-05-27T08:34:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 --2- 172.21.15.77:0/3788170470 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f516806bbe0 0x7f51681103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 --2- 172.21.15.77:0/3788170470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51681095b0 0x7f5168109990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 --2- 172.21.15.77:0/3788170470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f516806b220 0x7f516806b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 -- 172.21.15.77:0/3788170470 >> 172.21.15.77:0/3788170470 conn(0x7f51680793f0 msgr2=0x7f5168079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 -- 172.21.15.77:0/3788170470 shutdown_connections 2024-05-27T08:34:55.400 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.398+0000 7f5170654700 1 -- 172.21.15.77:0/3788170470 wait complete. 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f5170654700 1 Processor -- start 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f5170654700 1 -- start start 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f5170654700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f516806b220 0x7f516819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f5170654700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f516806bbe0 0x7f516819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f5170654700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51681095b0 0x7f51681a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f5170654700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51681136c0 con 0x7f516806b220 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f5170654700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5168113540 con 0x7f51681095b0 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f5170654700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5168113840 con 0x7f516806bbe0 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516e3f0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f516806b220 0x7f516819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:55.402 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516e3f0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f516806b220 0x7f516819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40802/0 (socket says 172.21.15.77:40802) 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516e3f0700 1 -- 172.21.15.77:0/2678437146 learned_addr learned my addr 172.21.15.77:0/2678437146 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516dbef700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f516806bbe0 0x7f516819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516ebf1700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51681095b0 0x7f51681a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516e3f0700 1 -- 172.21.15.77:0/2678437146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f516806bbe0 msgr2=0x7f516819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516e3f0700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f516806bbe0 0x7f516819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516e3f0700 1 -- 172.21.15.77:0/2678437146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51681095b0 msgr2=0x7f51681a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516e3f0700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51681095b0 0x7f51681a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.399+0000 7f516e3f0700 1 -- 172.21.15.77:0/2678437146 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51681a0db0 con 0x7f516806b220 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.400+0000 7f516e3f0700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f516806b220 0x7f516819bda0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5164000c00 tx=0x7f5164009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.400+0000 7f515f7fe700 1 -- 172.21.15.77:0/2678437146 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164027070 con 0x7f516806b220 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.400+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51681a1040 con 0x7f516806b220 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.400+0000 7f515f7fe700 1 -- 172.21.15.77:0/2678437146 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5164022470 con 0x7f516806b220 2024-05-27T08:34:55.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.400+0000 7f515f7fe700 1 -- 172.21.15.77:0/2678437146 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164017650 con 0x7f516806b220 2024-05-27T08:34:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.400+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51681a8920 con 0x7f516806b220 2024-05-27T08:34:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.401+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516806a230 con 0x7f516806b220 2024-05-27T08:34:55.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.402+0000 7f515f7fe700 1 -- 172.21.15.77:0/2678437146 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f516402a070 con 0x7f516806b220 2024-05-27T08:34:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.403+0000 7f515f7fe700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f51540b2900 0x7f51540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.404+0000 7f516dbef700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f51540b2900 0x7f51540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.404+0000 7f515f7fe700 1 -- 172.21.15.77:0/2678437146 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f51640e2f80 con 0x7f516806b220 2024-05-27T08:34:55.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.404+0000 7f516dbef700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f51540b2900 0x7f51540b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5168110810 tx=0x7f515800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:55.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.408+0000 7f515f7fe700 1 -- 172.21.15.77:0/2678437146 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5164013070 con 0x7f516806b220 2024-05-27T08:34:55.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.694+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51681a8af0 con 0x7f516806b220 2024-05-27T08:34:55.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.695+0000 7f515f7fe700 1 -- 172.21.15.77:0/2678437146 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7f51640ab130 con 0x7f516806b220 2024-05-27T08:34:55.696 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:55.697 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24410,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24412,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24434,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f51540b2900 msgr2=0x7f51540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f51540b2900 0x7f51540b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5168110810 tx=0x7f515800b410 comp rx=0 tx=0).stop 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f516806b220 msgr2=0x7f516819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f516806b220 0x7f516819bda0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5164000c00 tx=0x7f5164009580 comp rx=0 tx=0).stop 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 shutdown_connections 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f51681095b0 0x7f51681a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f516806b220 0x7f516819bda0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f51540b2900 0x7f51540b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 --2- 172.21.15.77:0/2678437146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f516806bbe0 0x7f516819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 >> 172.21.15.77:0/2678437146 conn(0x7f51680793f0 msgr2=0x7f5168100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 shutdown_connections 2024-05-27T08:34:55.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:55.697+0000 7f5170654700 1 -- 172.21.15.77:0/2678437146 wait complete. 2024-05-27T08:34:55.700 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 13 2024-05-27T08:34:56.730 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:56 smithi077 ceph-mon[32007]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:56.730 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.730 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.730 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.730 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:56 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.730 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:56 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2678437146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:56.732 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-27T08:34:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:56 smithi092 ceph-mon[36049]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:56 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:56 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2678437146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:56 smithi179 ceph-mon[36481]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:56 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:56.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:56 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2678437146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:34:56.991 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:34:58.432 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:58 smithi077 ceph-mon[32007]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:58.432 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:58.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 -- 172.21.15.77:0/329568440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c102810 msgr2=0x7fd63c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 --2- 172.21.15.77:0/329568440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c102810 0x7fd63c106810 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd63800aad0 tx=0x7fd63800ade0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 -- 172.21.15.77:0/329568440 shutdown_connections 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 --2- 172.21.15.77:0/329568440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd63c106ef0 0x7fd63c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 --2- 172.21.15.77:0/329568440 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd63c101e60 0x7fd63c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 --2- 172.21.15.77:0/329568440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c102810 0x7fd63c106810 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 -- 172.21.15.77:0/329568440 >> 172.21.15.77:0/329568440 conn(0x7fd63c078ab0 msgr2=0x7fd63c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 -- 172.21.15.77:0/329568440 shutdown_connections 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.453+0000 7fd6432b2700 1 -- 172.21.15.77:0/329568440 wait complete. 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd6432b2700 1 Processor -- start 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd6432b2700 1 -- start start 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd6432b2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd63c101e60 0x7fd63c0fdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:58.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd6432b2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd63c102810 0x7fd63c0fe460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd6432b2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c106ef0 0x7fd63c0fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd6432b2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c1114f0 con 0x7fd63c106ef0 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd6432b2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c111370 con 0x7fd63c101e60 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd6432b2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c111670 con 0x7fd63c102810 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd64184f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c106ef0 0x7fd63c0fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd64184f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c106ef0 0x7fd63c0fe9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40808/0 (socket says 172.21.15.77:40808) 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.454+0000 7fd64184f700 1 -- 172.21.15.77:0/1627540369 learned_addr learned my addr 172.21.15.77:0/1627540369 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64084d700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd63c102810 0x7fd63c0fe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64104e700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd63c101e60 0x7fd63c0fdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64184f700 1 -- 172.21.15.77:0/1627540369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd63c102810 msgr2=0x7fd63c0fe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64184f700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd63c102810 0x7fd63c0fe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64184f700 1 -- 172.21.15.77:0/1627540369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd63c101e60 msgr2=0x7fd63c0fdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64184f700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd63c101e60 0x7fd63c0fdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64184f700 1 -- 172.21.15.77:0/1627540369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c1a8170 con 0x7fd63c106ef0 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64104e700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd63c101e60 0x7fd63c0fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd64184f700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c106ef0 0x7fd63c0fe9a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd63400ada0 tx=0x7fd63400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd6327fc700 1 -- 172.21.15.77:0/1627540369 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634017020 con 0x7fd63c106ef0 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c1a8460 con 0x7fd63c106ef0 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd6327fc700 1 -- 172.21.15.77:0/1627540369 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd63400ce30 con 0x7fd63c106ef0 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c1a8970 con 0x7fd63c106ef0 2024-05-27T08:34:58.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.455+0000 7fd6327fc700 1 -- 172.21.15.77:0/1627540369 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6340054a0 con 0x7fd63c106ef0 2024-05-27T08:34:58.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.456+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd620005330 con 0x7fd63c106ef0 2024-05-27T08:34:58.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.458+0000 7fd6327fc700 1 -- 172.21.15.77:0/1627540369 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd634004ad0 con 0x7fd63c106ef0 2024-05-27T08:34:58.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.458+0000 7fd6327fc700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6280b2900 0x7fd6280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:34:58.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.459+0000 7fd64104e700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6280b2900 0x7fd6280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:34:58.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.459+0000 7fd6327fc700 1 -- 172.21.15.77:0/1627540369 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd6340e1960 con 0x7fd63c106ef0 2024-05-27T08:34:58.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.459+0000 7fd64104e700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6280b2900 0x7fd6280b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005fd0 tx=0x7fd62c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:34:58.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.462+0000 7fd6327fc700 1 -- 172.21.15.77:0/1627540369 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd634021020 con 0x7fd63c106ef0 2024-05-27T08:34:58.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:58 smithi092 ceph-mon[36049]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:58.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:58 smithi179 ceph-mon[36481]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:34:58.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:58.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.691+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd6200058f0 con 0x7fd63c106ef0 2024-05-27T08:34:58.693 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.692+0000 7fd6327fc700 1 -- 172.21.15.77:0/1627540369 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7fd6340a9bb0 con 0x7fd63c106ef0 2024-05-27T08:34:58.694 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:34:58.694 INFO:teuthology.orchestra.run.smithi077.stdout:[ 2024-05-27T08:34:58.694 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:34:58.694 INFO:teuthology.orchestra.run.smithi077.stdout: "poolnum": 1, 2024-05-27T08:34:58.694 INFO:teuthology.orchestra.run.smithi077.stdout: "poolname": ".mgr" 2024-05-27T08:34:58.694 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:34:58.694 INFO:teuthology.orchestra.run.smithi077.stdout:] 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6280b2900 msgr2=0x7fd6280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6280b2900 0x7fd6280b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005fd0 tx=0x7fd62c005e30 comp rx=0 tx=0).stop 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c106ef0 msgr2=0x7fd63c0fe9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c106ef0 0x7fd63c0fe9a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd63400ada0 tx=0x7fd63400c330 comp rx=0 tx=0).stop 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 shutdown_connections 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd63c106ef0 0x7fd63c0fe9a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd63c101e60 0x7fd63c0fdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6280b2900 0x7fd6280b4dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 --2- 172.21.15.77:0/1627540369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd63c102810 0x7fd63c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:34:58.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 >> 172.21.15.77:0/1627540369 conn(0x7fd63c078ab0 msgr2=0x7fd63c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:34:58.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.694+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 shutdown_connections 2024-05-27T08:34:58.697 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:34:58.695+0000 7fd6432b2700 1 -- 172.21.15.77:0/1627540369 wait complete. 2024-05-27T08:34:59.339 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-27T08:34:59.339 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-27T08:34:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:59 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1627540369' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T08:34:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:34:59 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:59.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:59.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:59.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:59 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1627540369' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T08:34:59.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:34:59 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:34:59.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:34:59.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:34:59.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:34:59.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:59 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1627540369' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-27T08:34:59.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:34:59 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:35:00.129 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:00.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:00 smithi077 ceph-mon[32007]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:00.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.769 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:00 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:00 smithi179 ceph-mon[36481]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:00.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:00 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:00 smithi092 ceph-mon[36049]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:00.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:00.987 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:00 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:01.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 -- 172.21.15.77:0/469843000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 msgr2=0x7fd30010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:01.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 --2- 172.21.15.77:0/469843000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 0x7fd30010f2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800b0a0 tx=0x7fd2f800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:35:01.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 -- 172.21.15.77:0/469843000 shutdown_connections 2024-05-27T08:35:01.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 --2- 172.21.15.77:0/469843000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 0x7fd30010f2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:01.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 --2- 172.21.15.77:0/469843000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300108520 0x7fd300108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:01.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 --2- 172.21.15.77:0/469843000 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd300102440 0x7fd3001028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:01.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 -- 172.21.15.77:0/469843000 >> 172.21.15.77:0/469843000 conn(0x7fd3000793f0 msgr2=0x7fd300079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:01.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 -- 172.21.15.77:0/469843000 shutdown_connections 2024-05-27T08:35:01.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.879+0000 7fd3054ca700 1 -- 172.21.15.77:0/469843000 wait complete. 2024-05-27T08:35:01.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd3054ca700 1 Processor -- start 2024-05-27T08:35:01.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd3054ca700 1 -- start start 2024-05-27T08:35:01.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd3054ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd300102440 0x7fd30019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:01.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd3054ca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 0x7fd30019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:01.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd3054ca700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300108520 0x7fd3001998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd3054ca700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3001125d0 con 0x7fd300102e00 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd3054ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd300112450 con 0x7fd300102440 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd3054ca700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd300112750 con 0x7fd300108520 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd2fe7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 0x7fd30019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd2fe7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 0x7fd30019fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47124/0 (socket says 172.21.15.77:47124) 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd2fe7fc700 1 -- 172.21.15.77:0/3951429568 learned_addr learned my addr 172.21.15.77:0/3951429568 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd2ff7fe700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300108520 0x7fd3001998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.880+0000 7fd2feffd700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd300102440 0x7fd30019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2fe7fc700 1 -- 172.21.15.77:0/3951429568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300108520 msgr2=0x7fd3001998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2fe7fc700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300108520 0x7fd3001998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2fe7fc700 1 -- 172.21.15.77:0/3951429568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd300102440 msgr2=0x7fd30019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2fe7fc700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd300102440 0x7fd30019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:01.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2fe7fc700 1 -- 172.21.15.77:0/3951429568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30019a050 con 0x7fd300102e00 2024-05-27T08:35:01.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2feffd700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd300102440 0x7fd30019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:35:01.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2fe7fc700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 0x7fd30019fc70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000c920 tx=0x7fd2f000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:01.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2e7fff700 1 -- 172.21.15.77:0/3951429568 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0007b00 con 0x7fd300102e00 2024-05-27T08:35:01.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30019a340 con 0x7fd300102e00 2024-05-27T08:35:01.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2e7fff700 1 -- 172.21.15.77:0/3951429568 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2f0004ce0 con 0x7fd300102e00 2024-05-27T08:35:01.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd2e7fff700 1 -- 172.21.15.77:0/3951429568 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0005500 con 0x7fd300102e00 2024-05-27T08:35:01.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.881+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3001a6600 con 0x7fd300102e00 2024-05-27T08:35:01.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.882+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd30006a230 con 0x7fd300102e00 2024-05-27T08:35:01.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.883+0000 7fd2e7fff700 1 -- 172.21.15.77:0/3951429568 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd2f0007ca0 con 0x7fd300102e00 2024-05-27T08:35:01.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.884+0000 7fd2e7fff700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd2e80b2880 0x7fd2e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:01.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.885+0000 7fd2feffd700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd2e80b2880 0x7fd2e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:01.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.885+0000 7fd2e7fff700 1 -- 172.21.15.77:0/3951429568 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd2f00e1910 con 0x7fd300102e00 2024-05-27T08:35:01.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.885+0000 7fd2feffd700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd2e80b2880 0x7fd2e80b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4005ea0 tx=0x7fd2f4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:01.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:01.891+0000 7fd2e7fff700 1 -- 172.21.15.77:0/3951429568 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd2f0014030 con 0x7fd300102e00 2024-05-27T08:35:02.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:02.117+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd30002e560 con 0x7fd300102e00 2024-05-27T08:35:03.001 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.000+0000 7fd2e7fff700 1 -- 172.21.15.77:0/3951429568 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fd2f00a9ad0 con 0x7fd300102e00 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd2e80b2880 msgr2=0x7fd2e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd2e80b2880 0x7fd2e80b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4005ea0 tx=0x7fd2f4005e30 comp rx=0 tx=0).stop 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 msgr2=0x7fd30019fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 0x7fd30019fc70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000c920 tx=0x7fd2f000cc30 comp rx=0 tx=0).stop 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 shutdown_connections 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd2e80b2880 0x7fd2e80b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300108520 0x7fd3001998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd300102440 0x7fd30019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 --2- 172.21.15.77:0/3951429568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd300102e00 0x7fd30019fc70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:03.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.003+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 >> 172.21.15.77:0/3951429568 conn(0x7fd3000793f0 msgr2=0x7fd30006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.004+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 shutdown_connections 2024-05-27T08:35:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:03.004+0000 7fd3054ca700 1 -- 172.21.15.77:0/3951429568 wait complete. 2024-05-27T08:35:03.006 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_metadata' created 2024-05-27T08:35:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:02 smithi179 ceph-mon[36481]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:02 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3951429568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T08:35:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:35:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:35:03.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:02 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:02 smithi077 ceph-mon[32007]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:03.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.363 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:02 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3951429568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T08:35:03.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:35:03.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:35:03.364 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:02 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:02 smithi092 ceph-mon[36049]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:03.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:02 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3951429568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-27T08:35:03.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:35:03.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:35:03.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:02 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:35:03.652 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-27T08:35:03.896 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:04.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:04 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3951429568' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T08:35:04.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:04 smithi077 ceph-mon[32007]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T08:35:04.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:04 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3951429568' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T08:35:04.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:04 smithi179 ceph-mon[36481]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T08:35:04.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:04 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3951429568' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-27T08:35:04.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:04 smithi092 ceph-mon[36049]: osdmap e77: 12 total, 12 up, 12 in 2024-05-27T08:35:05.306 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:05 smithi077 ceph-mon[32007]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:05.306 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:05 smithi077 ceph-mon[32007]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T08:35:05.306 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:05 smithi077 ceph-mon[32007]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T08:35:05.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:05 smithi179 ceph-mon[36481]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:05.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:05 smithi179 ceph-mon[36481]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T08:35:05.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:05 smithi179 ceph-mon[36481]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T08:35:05.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.413+0000 7f46dfecc700 1 -- 172.21.15.77:0/1163006481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102e30 msgr2=0x7f46d810f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:05.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.413+0000 7f46dfecc700 1 --2- 172.21.15.77:0/1163006481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102e30 0x7f46d810f2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f46d000b0a0 tx=0x7f46d000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:35:05.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.413+0000 7f46dfecc700 1 -- 172.21.15.77:0/1163006481 shutdown_connections 2024-05-27T08:35:05.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.413+0000 7f46dfecc700 1 --2- 172.21.15.77:0/1163006481 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102e30 0x7f46d810f2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:05.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.413+0000 7f46dfecc700 1 --2- 172.21.15.77:0/1163006481 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d81084b0 0x7f46d8108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:05.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.413+0000 7f46dfecc700 1 --2- 172.21.15.77:0/1163006481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46d8102470 0x7f46d81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:05.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.413+0000 7f46dfecc700 1 -- 172.21.15.77:0/1163006481 >> 172.21.15.77:0/1163006481 conn(0x7f46d8079410 msgr2=0x7f46d8079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:05.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 -- 172.21.15.77:0/1163006481 shutdown_connections 2024-05-27T08:35:05.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 -- 172.21.15.77:0/1163006481 wait complete. 2024-05-27T08:35:05.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 Processor -- start 2024-05-27T08:35:05.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 -- start start 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102470 0x7f46d819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46d8102e30 0x7f46d819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d81084b0 0x7f46d81a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81125f0 con 0x7f46d8102470 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8112470 con 0x7f46d81084b0 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.414+0000 7f46dfecc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8112770 con 0x7f46d8102e30 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102470 0x7f46d819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102470 0x7f46d819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47146/0 (socket says 172.21.15.77:47146) 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 -- 172.21.15.77:0/981049902 learned_addr learned my addr 172.21.15.77:0/981049902 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46dd467700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46d8102e30 0x7f46d819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46de469700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d81084b0 0x7f46d81a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:05.418 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 -- 172.21.15.77:0/981049902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46d8102e30 msgr2=0x7f46d819af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46d8102e30 0x7f46d819af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 -- 172.21.15.77:0/981049902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d81084b0 msgr2=0x7f46d81a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d81084b0 0x7f46d81a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 -- 172.21.15.77:0/981049902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d81a5fb0 con 0x7f46d8102470 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ddc68700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102470 0x7f46d819a9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46d400d950 tx=0x7f46d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ceffd700 1 -- 172.21.15.77:0/981049902 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4016070 con 0x7f46d8102470 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ceffd700 1 -- 172.21.15.77:0/981049902 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46d400fd40 con 0x7f46d8102470 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d81a61e0 con 0x7f46d8102470 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46ceffd700 1 -- 172.21.15.77:0/981049902 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d400e3b0 con 0x7f46d8102470 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.415+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d81a66f0 con 0x7f46d8102470 2024-05-27T08:35:05.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.416+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d806a230 con 0x7f46d8102470 2024-05-27T08:35:05.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.418+0000 7f46ceffd700 1 -- 172.21.15.77:0/981049902 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f46d4019070 con 0x7f46d8102470 2024-05-27T08:35:05.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.418+0000 7f46ceffd700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f46c40b2880 0x7f46c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:05.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.419+0000 7f46dd467700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f46c40b2880 0x7f46c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:05.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.419+0000 7f46ceffd700 1 -- 172.21.15.77:0/981049902 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f46d40e18f0 con 0x7f46d8102470 2024-05-27T08:35:05.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.419+0000 7f46dd467700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f46c40b2880 0x7f46c40b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f46d80799f0 tx=0x7f46c800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.422+0000 7f46ceffd700 1 -- 172.21.15.77:0/981049902 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46d40a9940 con 0x7f46d8102470 2024-05-27T08:35:05.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:05 smithi092 ceph-mon[36049]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:05.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:05 smithi092 ceph-mon[36049]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-27T08:35:05.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:05 smithi092 ceph-mon[36049]: osdmap e78: 12 total, 12 up, 12 in 2024-05-27T08:35:05.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:05.628+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f46d802e720 con 0x7f46d8102470 2024-05-27T08:35:06.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.023+0000 7f46ceffd700 1 -- 172.21.15.77:0/981049902 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7f46d40a9240 con 0x7f46d8102470 2024-05-27T08:35:06.027 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.025+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f46c40b2880 msgr2=0x7f46c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.025+0000 7f46dfecc700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f46c40b2880 0x7f46c40b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f46d80799f0 tx=0x7f46c800b3c0 comp rx=0 tx=0).stop 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.025+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102470 msgr2=0x7f46d819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.025+0000 7f46dfecc700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102470 0x7f46d819a9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f46d400d950 tx=0x7f46d400dc60 comp rx=0 tx=0).stop 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.026+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 shutdown_connections 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.026+0000 7f46dfecc700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46d81084b0 0x7f46d81a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.026+0000 7f46dfecc700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46d8102470 0x7f46d819a9f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.026+0000 7f46dfecc700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f46c40b2880 0x7f46c40b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.026+0000 7f46dfecc700 1 --2- 172.21.15.77:0/981049902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46d8102e30 0x7f46d819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:06.028 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.026+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 >> 172.21.15.77:0/981049902 conn(0x7f46d8079410 msgr2=0x7f46d8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.026+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 shutdown_connections 2024-05-27T08:35:06.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:06.026+0000 7f46dfecc700 1 -- 172.21.15.77:0/981049902 wait complete. 2024-05-27T08:35:06.030 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data' created 2024-05-27T08:35:06.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:06 smithi092 ceph-mon[36049]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T08:35:06.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:06 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/981049902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T08:35:06.343 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:06 smithi077 ceph-mon[32007]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T08:35:06.343 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:06 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/981049902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T08:35:06.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:06 smithi179 ceph-mon[36481]: osdmap e79: 12 total, 12 up, 12 in 2024-05-27T08:35:06.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:06 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/981049902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-27T08:35:06.679 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-27T08:35:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:07.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:07 smithi179 ceph-mon[36481]: pgmap v283: 33 pgs: 1 creating+activating, 11 creating+peering, 8 active+clean, 13 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:07.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:07 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/981049902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T08:35:07.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:07 smithi179 ceph-mon[36481]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T08:35:07.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:07 smithi077 ceph-mon[32007]: pgmap v283: 33 pgs: 1 creating+activating, 11 creating+peering, 8 active+clean, 13 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:07.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:07 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/981049902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T08:35:07.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:07 smithi077 ceph-mon[32007]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T08:35:07.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:07 smithi092 ceph-mon[36049]: pgmap v283: 33 pgs: 1 creating+activating, 11 creating+peering, 8 active+clean, 13 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:07.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:07 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/981049902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-27T08:35:07.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:07 smithi092 ceph-mon[36049]: osdmap e80: 12 total, 12 up, 12 in 2024-05-27T08:35:08.239 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:08 smithi077 ceph-mon[32007]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T08:35:08.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:08 smithi179 ceph-mon[36481]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T08:35:08.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:08 smithi092 ceph-mon[36049]: osdmap e81: 12 total, 12 up, 12 in 2024-05-27T08:35:08.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 -- 172.21.15.77:0/3690240699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b441039b0 msgr2=0x7f0b4410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:08.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 --2- 172.21.15.77:0/3690240699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b441039b0 0x7f0b4410f3c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00b0d0 tx=0x7f0b3c00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:35:08.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 -- 172.21.15.77:0/3690240699 shutdown_connections 2024-05-27T08:35:08.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 --2- 172.21.15.77:0/3690240699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b441039b0 0x7f0b4410f3c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:08.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 --2- 172.21.15.77:0/3690240699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b44108710 0x7f0b44108af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:08.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 --2- 172.21.15.77:0/3690240699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b44102ff0 0x7f0b44103470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:08.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 -- 172.21.15.77:0/3690240699 >> 172.21.15.77:0/3690240699 conn(0x7f0b44077760 msgr2=0x7f0b44077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:08.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 -- 172.21.15.77:0/3690240699 shutdown_connections 2024-05-27T08:35:08.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 -- 172.21.15.77:0/3690240699 wait complete. 2024-05-27T08:35:08.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.507+0000 7f0b49926700 1 Processor -- start 2024-05-27T08:35:08.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b49926700 1 -- start start 2024-05-27T08:35:08.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b49926700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b44102ff0 0x7f0b4419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:08.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b49926700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b441039b0 0x7f0b4419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b49926700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b44108710 0x7f0b441a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b49926700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44112790 con 0x7f0b44108710 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b49926700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44112610 con 0x7f0b441039b0 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b49926700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44112910 con 0x7f0b44102ff0 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b44108710 0x7f0b441a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b42ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b44102ff0 0x7f0b4419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b437fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b44108710 0x7f0b441a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47166/0 (socket says 172.21.15.77:47166) 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b42ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b44102ff0 0x7f0b4419bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.77:41760/0 (socket says 172.21.15.77:41760) 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b437fe700 1 -- 172.21.15.77:0/120006493 learned_addr learned my addr 172.21.15.77:0/120006493 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b427fc700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b441039b0 0x7f0b4419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:08.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b437fe700 1 -- 172.21.15.77:0/120006493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b44102ff0 msgr2=0x7f0b4419bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b437fe700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b44102ff0 0x7f0b4419bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b437fe700 1 -- 172.21.15.77:0/120006493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b441039b0 msgr2=0x7f0b4419c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b437fe700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b441039b0 0x7f0b4419c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b437fe700 1 -- 172.21.15.77:0/120006493 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b441a0f50 con 0x7f0b44108710 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.508+0000 7f0b42ffd700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b44102ff0 0x7f0b4419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.509+0000 7f0b437fe700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b44108710 0x7f0b441a0810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009fd0 tx=0x7f0b3c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.509+0000 7f0b48924700 1 -- 172.21.15.77:0/120006493 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c01f070 con 0x7f0b44108710 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.509+0000 7f0b48924700 1 -- 172.21.15.77:0/120006493 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b3c007e40 con 0x7f0b44108710 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.509+0000 7f0b48924700 1 -- 172.21.15.77:0/120006493 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c01b460 con 0x7f0b44108710 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.509+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b441a11e0 con 0x7f0b44108710 2024-05-27T08:35:08.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.509+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b441a8ac0 con 0x7f0b44108710 2024-05-27T08:35:08.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.510+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b4406a230 con 0x7f0b44108710 2024-05-27T08:35:08.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.512+0000 7f0b48924700 1 -- 172.21.15.77:0/120006493 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0b3c022070 con 0x7f0b44108710 2024-05-27T08:35:08.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.513+0000 7f0b48924700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0b2c0b2880 0x7f0b2c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:08.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.513+0000 7f0b42ffd700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0b2c0b2880 0x7f0b2c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:08.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.513+0000 7f0b48924700 1 -- 172.21.15.77:0/120006493 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7879+0+0 (secure 0 0 0) 0x7f0b3c00f020 con 0x7f0b44108710 2024-05-27T08:35:08.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.513+0000 7f0b42ffd700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0b2c0b2880 0x7f0b2c0b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b38005950 tx=0x7f0b3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:08.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.517+0000 7f0b48924700 1 -- 172.21.15.77:0/120006493 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0b3c029020 con 0x7f0b44108710 2024-05-27T08:35:08.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:08.764+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0b4402e560 con 0x7f0b44108710 2024-05-27T08:35:09.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.038+0000 7f0b48924700 1 -- 172.21.15.77:0/120006493 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f0b3c0ab490 con 0x7f0b44108710 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0b2c0b2880 msgr2=0x7f0b2c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0b2c0b2880 0x7f0b2c0b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b38005950 tx=0x7f0b3800b410 comp rx=0 tx=0).stop 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b44108710 msgr2=0x7f0b441a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b44108710 0x7f0b441a0810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c009fd0 tx=0x7f0b3c007a00 comp rx=0 tx=0).stop 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 shutdown_connections 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b44108710 0x7f0b441a0810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f0b2c0b2880 0x7f0b2c0b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b44102ff0 0x7f0b4419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 --2- 172.21.15.77:0/120006493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b441039b0 0x7f0b4419c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 >> 172.21.15.77:0/120006493 conn(0x7f0b44077760 msgr2=0x7f0b4410d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 shutdown_connections 2024-05-27T08:35:09.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:09.041+0000 7f0b49926700 1 -- 172.21.15.77:0/120006493 wait complete. 2024-05-27T08:35:09.045 INFO:teuthology.orchestra.run.smithi077.stderr:new fs with metadata pool 2 and data pool 3 2024-05-27T08:35:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:09 smithi179 ceph-mon[36481]: pgmap v286: 97 pgs: 1 creating+activating, 33 creating+peering, 12 active+clean, 51 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:09 smithi179 ceph-mon[36481]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T08:35:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:09 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/120006493' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T08:35:09.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:09 smithi179 ceph-mon[36481]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T08:35:09.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:09 smithi077 ceph-mon[32007]: pgmap v286: 97 pgs: 1 creating+activating, 33 creating+peering, 12 active+clean, 51 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:09.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:09 smithi077 ceph-mon[32007]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T08:35:09.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:09 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/120006493' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T08:35:09.365 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:09 smithi077 ceph-mon[32007]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T08:35:09.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:09 smithi092 ceph-mon[36049]: pgmap v286: 97 pgs: 1 creating+activating, 33 creating+peering, 12 active+clean, 51 unknown; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:09.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:09 smithi092 ceph-mon[36049]: osdmap e82: 12 total, 12 up, 12 in 2024-05-27T08:35:09.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:09 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/120006493' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-27T08:35:09.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:09 smithi092 ceph-mon[36049]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-27T08:35:09.652 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:35:09.893 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:10.319 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:10 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/120006493' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T08:35:10.320 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:10 smithi077 ceph-mon[32007]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T08:35:10.320 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:10 smithi077 ceph-mon[32007]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T08:35:10.320 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:10 smithi077 ceph-mon[32007]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T08:35:10.320 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:10 smithi077 ceph-mon[32007]: pgmap v289: 97 pgs: 22 creating+peering, 67 active+clean, 8 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:10.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:10 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/120006493' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T08:35:10.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:10 smithi179 ceph-mon[36481]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T08:35:10.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:10 smithi179 ceph-mon[36481]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T08:35:10.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:10 smithi179 ceph-mon[36481]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T08:35:10.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:10 smithi179 ceph-mon[36481]: pgmap v289: 97 pgs: 22 creating+peering, 67 active+clean, 8 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:10.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:10 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/120006493' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-27T08:35:10.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:10 smithi092 ceph-mon[36049]: osdmap e83: 12 total, 12 up, 12 in 2024-05-27T08:35:10.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:10 smithi092 ceph-mon[36049]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-27T08:35:10.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:10 smithi092 ceph-mon[36049]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-27T08:35:10.388 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:10 smithi092 ceph-mon[36049]: pgmap v289: 97 pgs: 22 creating+peering, 67 active+clean, 8 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:35:11.234 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:11 smithi077 ceph-mon[32007]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T08:35:11.234 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:11 smithi077 ceph-mon[32007]: Cluster is now healthy 2024-05-27T08:35:11.234 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:11 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] up:active 2024-05-27T08:35:11.235 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:11 smithi077 ceph-mon[32007]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:11.235 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:11 smithi077 ceph-mon[32007]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:11.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:11 smithi179 ceph-mon[36481]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T08:35:11.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:11 smithi179 ceph-mon[36481]: Cluster is now healthy 2024-05-27T08:35:11.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:11 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] up:active 2024-05-27T08:35:11.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:11 smithi179 ceph-mon[36481]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:11.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:11 smithi179 ceph-mon[36481]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 -- 172.21.15.77:0/2811865372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90102440 msgr2=0x7f5b901028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/2811865372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90102440 0x7f5b901028c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009990 tx=0x7f5b80009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 -- 172.21.15.77:0/2811865372 shutdown_connections 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/2811865372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b90102e00 0x7f5b9010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/2811865372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b90108520 0x7f5b90108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/2811865372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90102440 0x7f5b901028c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 -- 172.21.15.77:0/2811865372 >> 172.21.15.77:0/2811865372 conn(0x7f5b900793f0 msgr2=0x7f5b90079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 -- 172.21.15.77:0/2811865372 shutdown_connections 2024-05-27T08:35:11.448 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.443+0000 7f5b94b1d700 1 -- 172.21.15.77:0/2811865372 wait complete. 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b94b1d700 1 Processor -- start 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b94b1d700 1 -- start start 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b94b1d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b90102440 0x7f5b9019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b94b1d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b90102e00 0x7f5b9019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b94b1d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90108520 0x7f5b90198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b94b1d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b90112650 con 0x7f5b90108520 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b94b1d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901124d0 con 0x7f5b90102440 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b94b1d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b901127d0 con 0x7f5b90102e00 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b8ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90108520 0x7f5b90198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b8ed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90108520 0x7f5b90198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50570/0 (socket says 172.21.15.77:50570) 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b8ed9d700 1 -- 172.21.15.77:0/765998126 learned_addr learned my addr 172.21.15.77:0/765998126 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:11.449 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b8e59c700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b90102440 0x7f5b9019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.444+0000 7f5b8dd9b700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b90102e00 0x7f5b9019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b8ed9d700 1 -- 172.21.15.77:0/765998126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b90102e00 msgr2=0x7f5b9019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b8ed9d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b90102e00 0x7f5b9019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b8ed9d700 1 -- 172.21.15.77:0/765998126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b90102440 msgr2=0x7f5b9019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b8ed9d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b90102440 0x7f5b9019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b8ed9d700 1 -- 172.21.15.77:0/765998126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b901990f0 con 0x7f5b90108520 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b8e59c700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b90102440 0x7f5b9019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b8ed9d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90108520 0x7f5b90198990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800ef30 tx=0x7f5b8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b877fe700 1 -- 172.21.15.77:0/765998126 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880092e0 con 0x7f5b90108520 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b90199380 con 0x7f5b90108520 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b877fe700 1 -- 172.21.15.77:0/765998126 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b88004bb0 con 0x7f5b90108520 2024-05-27T08:35:11.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b877fe700 1 -- 172.21.15.77:0/765998126 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88004020 con 0x7f5b90108520 2024-05-27T08:35:11.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.445+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b901a56a0 con 0x7f5b90108520 2024-05-27T08:35:11.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.446+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9010a9b0 con 0x7f5b90108520 2024-05-27T08:35:11.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.447+0000 7f5b877fe700 1 -- 172.21.15.77:0/765998126 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5b8801b030 con 0x7f5b90108520 2024-05-27T08:35:11.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.448+0000 7f5b877fe700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b7c0b2900 0x7f5b7c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:11.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.448+0000 7f5b8e59c700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b7c0b2900 0x7f5b7c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:11.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.448+0000 7f5b877fe700 1 -- 172.21.15.77:0/765998126 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f5b880e2520 con 0x7f5b90108520 2024-05-27T08:35:11.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.449+0000 7f5b8e59c700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b7c0b2900 0x7f5b7c0b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5b90199ad0 tx=0x7f5b78009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:11.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.452+0000 7f5b877fe700 1 -- 172.21.15.77:0/765998126 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b88022020 con 0x7f5b90108520 2024-05-27T08:35:11.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:11 smithi092 ceph-mon[36049]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-27T08:35:11.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:11 smithi092 ceph-mon[36049]: Cluster is now healthy 2024-05-27T08:35:11.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:11 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] up:active 2024-05-27T08:35:11.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:11 smithi092 ceph-mon[36049]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:11.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:11 smithi092 ceph-mon[36049]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:11.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.657+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b90199ec0 con 0x7f5b90108520 2024-05-27T08:35:11.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.658+0000 7f5b877fe700 1 -- 172.21.15.77:0/765998126 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22195 (secure 0 0 0) 0x7f5b88010040 con 0x7f5b90108520 2024-05-27T08:35:11.661 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:35:11.661 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":83,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:35:08.767247+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:35:11.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b7c0b2900 msgr2=0x7f5b7c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:11.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b7c0b2900 0x7f5b7c0b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5b90199ad0 tx=0x7f5b78009190 comp rx=0 tx=0).stop 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90108520 msgr2=0x7f5b90198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90108520 0x7f5b90198990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800ef30 tx=0x7f5b8800c3a0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 shutdown_connections 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b90108520 0x7f5b90198990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b90102440 0x7f5b9019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b7c0b2900 0x7f5b7c0b4dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 --2- 172.21.15.77:0/765998126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b90102e00 0x7f5b9019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 >> 172.21.15.77:0/765998126 conn(0x7f5b900793f0 msgr2=0x7f5b901005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 shutdown_connections 2024-05-27T08:35:11.664 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:11.661+0000 7f5b94b1d700 1 -- 172.21.15.77:0/765998126 wait complete. 2024-05-27T08:35:12.290 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:12 smithi077 ceph-mon[32007]: pgmap v290: 97 pgs: 14 creating+peering, 83 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2024-05-27T08:35:12.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:12 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/765998126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:12.319 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:35:12.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:12 smithi179 ceph-mon[36481]: pgmap v290: 97 pgs: 14 creating+peering, 83 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2024-05-27T08:35:12.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:12 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/765998126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:12.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:12 smithi092 ceph-mon[36049]: pgmap v290: 97 pgs: 14 creating+peering, 83 active+clean; 578 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 1 op/s 2024-05-27T08:35:12.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:12 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/765998126' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:12.567 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:14.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 -- 172.21.15.77:0/379681514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c1084b0 msgr2=0x7f4c5c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:14.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 --2- 172.21.15.77:0/379681514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c1084b0 0x7f4c5c108890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009990 tx=0x7f4c4c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.178 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 -- 172.21.15.77:0/379681514 shutdown_connections 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 --2- 172.21.15.77:0/379681514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c5c102e30 0x7f4c5c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 --2- 172.21.15.77:0/379681514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c1084b0 0x7f4c5c108890 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 --2- 172.21.15.77:0/379681514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c5c102470 0x7f4c5c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 -- 172.21.15.77:0/379681514 >> 172.21.15.77:0/379681514 conn(0x7f4c5c0793f0 msgr2=0x7f4c5c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 -- 172.21.15.77:0/379681514 shutdown_connections 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 -- 172.21.15.77:0/379681514 wait complete. 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.175+0000 7f4c61815700 1 Processor -- start 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c61815700 1 -- start start 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c61815700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c5c102470 0x7f4c5c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c61815700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c102e30 0x7f4c5c19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c61815700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c5c1084b0 0x7f4c5c1a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c61815700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c112640 con 0x7f4c5c102e30 2024-05-27T08:35:14.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c61815700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1124c0 con 0x7f4c5c102470 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c61815700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1127c0 con 0x7f4c5c1084b0 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c102e30 0x7f4c5c19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c102e30 0x7f4c5c19b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50590/0 (socket says 172.21.15.77:50590) 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 -- 172.21.15.77:0/2110069071 learned_addr learned my addr 172.21.15.77:0/2110069071 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5b7fe700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c5c1084b0 0x7f4c5c1a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5affd700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c5c102470 0x7f4c5c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 -- 172.21.15.77:0/2110069071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c5c1084b0 msgr2=0x7f4c5c1a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c5c1084b0 0x7f4c5c1a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 -- 172.21.15.77:0/2110069071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c5c102470 msgr2=0x7f4c5c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c5c102470 0x7f4c5c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 -- 172.21.15.77:0/2110069071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c1a60a0 con 0x7f4c5c102e30 2024-05-27T08:35:14.180 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.176+0000 7f4c5a7fc700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c102e30 0x7f4c5c19b020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000d950 tx=0x7f4c5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.177+0000 7f4c60813700 1 -- 172.21.15.77:0/2110069071 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c50016070 con 0x7f4c5c102e30 2024-05-27T08:35:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.177+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c1a62d0 con 0x7f4c5c102e30 2024-05-27T08:35:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.177+0000 7f4c60813700 1 -- 172.21.15.77:0/2110069071 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5000fd40 con 0x7f4c5c102e30 2024-05-27T08:35:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.177+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c1a6830 con 0x7f4c5c102e30 2024-05-27T08:35:14.181 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.177+0000 7f4c60813700 1 -- 172.21.15.77:0/2110069071 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000e3b0 con 0x7f4c5c102e30 2024-05-27T08:35:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.178+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c06a230 con 0x7f4c5c102e30 2024-05-27T08:35:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.179+0000 7f4c60813700 1 -- 172.21.15.77:0/2110069071 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4c50019070 con 0x7f4c5c102e30 2024-05-27T08:35:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.180+0000 7f4c60813700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4c440b2880 0x7f4c440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.180+0000 7f4c5affd700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4c440b2880 0x7f4c440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.180+0000 7f4c60813700 1 -- 172.21.15.77:0/2110069071 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f4c500e1fd0 con 0x7f4c5c102e30 2024-05-27T08:35:14.182 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.180+0000 7f4c5affd700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4c440b2880 0x7f4c440b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0098e0 tx=0x7f4c4c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:14.184 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.183+0000 7f4c60813700 1 -- 172.21.15.77:0/2110069071 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4c500a9e40 con 0x7f4c5c102e30 2024-05-27T08:35:14.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.392+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c5c06c260 con 0x7f4c5c102e30 2024-05-27T08:35:14.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.393+0000 7f4c60813700 1 -- 172.21.15.77:0/2110069071 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22195 (secure 0 0 0) 0x7f4c500a9740 con 0x7f4c5c102e30 2024-05-27T08:35:14.395 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:35:14.395 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":83,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:35:08.767247+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.395+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4c440b2880 msgr2=0x7f4c440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.395+0000 7f4c61815700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4c440b2880 0x7f4c440b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0098e0 tx=0x7f4c4c005960 comp rx=0 tx=0).stop 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.395+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c102e30 msgr2=0x7f4c5c19b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.395+0000 7f4c61815700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c102e30 0x7f4c5c19b020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4c5000d950 tx=0x7f4c5000dc60 comp rx=0 tx=0).stop 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.396+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 shutdown_connections 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.396+0000 7f4c61815700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4c440b2880 0x7f4c440b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.396+0000 7f4c61815700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c5c1084b0 0x7f4c5c1a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.396+0000 7f4c61815700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4c5c102470 0x7f4c5c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.396+0000 7f4c61815700 1 --2- 172.21.15.77:0/2110069071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c5c102e30 0x7f4c5c19b020 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:14.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.396+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 >> 172.21.15.77:0/2110069071 conn(0x7f4c5c0793f0 msgr2=0x7f4c5c100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:14.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.396+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 shutdown_connections 2024-05-27T08:35:14.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:14.396+0000 7f4c61815700 1 -- 172.21.15.77:0/2110069071 wait complete. 2024-05-27T08:35:14.486 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:35:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: 2024-05-27T08:35:14.223+0000 7f2d3a3fc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-27T08:35:14.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:14 smithi092 ceph-mon[36049]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-27T08:35:14.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:14 smithi179 ceph-mon[36481]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-27T08:35:14.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:14 smithi077 ceph-mon[32007]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-05-27T08:35:14.948 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-27T08:35:15.193 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:15.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:15 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2110069071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:15.513 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:15 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2110069071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:15.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:15 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2110069071' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:16.468 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:16 smithi077 ceph-mon[32007]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T08:35:16.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:16 smithi092 ceph-mon[36049]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T08:35:16.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:16 smithi179 ceph-mon[36481]: pgmap v292: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-27T08:35:16.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.775+0000 7f99dc219700 1 -- 172.21.15.77:0/4080240347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 msgr2=0x7f99d4108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:16.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.775+0000 7f99dc219700 1 --2- 172.21.15.77:0/4080240347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 0x7f99d4108890 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009990 tx=0x7f99c4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:16.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.775+0000 7f99dc219700 1 -- 172.21.15.77:0/4080240347 shutdown_connections 2024-05-27T08:35:16.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.775+0000 7f99dc219700 1 --2- 172.21.15.77:0/4080240347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99d4102e30 0x7f99d410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:16.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.775+0000 7f99dc219700 1 --2- 172.21.15.77:0/4080240347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 0x7f99d4108890 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:16.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.775+0000 7f99dc219700 1 --2- 172.21.15.77:0/4080240347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99d4102470 0x7f99d41028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:16.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 -- 172.21.15.77:0/4080240347 >> 172.21.15.77:0/4080240347 conn(0x7f99d4079410 msgr2=0x7f99d4079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 -- 172.21.15.77:0/4080240347 shutdown_connections 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 -- 172.21.15.77:0/4080240347 wait complete. 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 Processor -- start 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 -- start start 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99d4102470 0x7f99d4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99d4102e30 0x7f99d419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 0x7f99d419e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99d41126c0 con 0x7f99d41084b0 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f99d4112540 con 0x7f99d4102e30 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.776+0000 7f99dc219700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f99d4112840 con 0x7f99d4102470 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 0x7f99d419e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 0x7f99d419e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50612/0 (socket says 172.21.15.77:50612) 2024-05-27T08:35:16.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 -- 172.21.15.77:0/1111949371 learned_addr learned my addr 172.21.15.77:0/1111949371 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99d97b4700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99d4102e30 0x7f99d419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99d9fb5700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99d4102470 0x7f99d4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 -- 172.21.15.77:0/1111949371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99d4102470 msgr2=0x7f99d4199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99d4102470 0x7f99d4199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 -- 172.21.15.77:0/1111949371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99d4102e30 msgr2=0x7f99d419a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99d4102e30 0x7f99d419a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 -- 172.21.15.77:0/1111949371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d419edb0 con 0x7f99d41084b0 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99da7b6700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 0x7f99d419e670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00b970 tx=0x7f99cc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99caffd700 1 -- 172.21.15.77:0/1111949371 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00d860 con 0x7f99d41084b0 2024-05-27T08:35:16.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d419f0a0 con 0x7f99d41084b0 2024-05-27T08:35:16.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99caffd700 1 -- 172.21.15.77:0/1111949371 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99cc00e430 con 0x7f99d41084b0 2024-05-27T08:35:16.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99caffd700 1 -- 172.21.15.77:0/1111949371 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc017660 con 0x7f99d41084b0 2024-05-27T08:35:16.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.777+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d41a6920 con 0x7f99d41084b0 2024-05-27T08:35:16.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.778+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d406a230 con 0x7f99d41084b0 2024-05-27T08:35:16.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.780+0000 7f99caffd700 1 -- 172.21.15.77:0/1111949371 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f99cc00da00 con 0x7f99d41084b0 2024-05-27T08:35:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.780+0000 7f99caffd700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f99c00b2880 0x7f99c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.781+0000 7f99d9fb5700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f99c00b2880 0x7f99c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.781+0000 7f99caffd700 1 -- 172.21.15.77:0/1111949371 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f99cc0e20a0 con 0x7f99d41084b0 2024-05-27T08:35:16.784 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.781+0000 7f99d9fb5700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f99c00b2880 0x7f99c00b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f99c400b5c0 tx=0x7f99c4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:16.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:16.784+0000 7f99caffd700 1 -- 172.21.15.77:0/1111949371 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f99cc013030 con 0x7f99d41084b0 2024-05-27T08:35:17.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.032+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f99d41a6b50 con 0x7f99d41084b0 2024-05-27T08:35:17.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.212+0000 7f99caffd700 1 -- 172.21.15.77:0/1111949371 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f99cc0a9f10 con 0x7f99d41084b0 2024-05-27T08:35:17.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f99c00b2880 msgr2=0x7f99c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:17.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f99c00b2880 0x7f99c00b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f99c400b5c0 tx=0x7f99c4005960 comp rx=0 tx=0).stop 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 msgr2=0x7f99d419e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 0x7f99d419e670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00b970 tx=0x7f99cc00bc80 comp rx=0 tx=0).stop 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 shutdown_connections 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99d41084b0 0x7f99d419e670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f99c00b2880 0x7f99c00b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99d4102470 0x7f99d4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 --2- 172.21.15.77:0/1111949371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f99d4102e30 0x7f99d419a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 >> 172.21.15.77:0/1111949371 conn(0x7f99d4079410 msgr2=0x7f99d4106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 shutdown_connections 2024-05-27T08:35:17.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:17.215+0000 7f99dc219700 1 -- 172.21.15.77:0/1111949371 wait complete. 2024-05-27T08:35:17.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:17 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1111949371' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T08:35:17.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:17 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1111949371' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T08:35:17.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:17 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1111949371' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-27T08:35:17.838 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-27T08:35:17.838 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-27T08:35:18.080 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:18.411 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:18 smithi077 ceph-mon[32007]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-27T08:35:18.411 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:18 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1111949371' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T08:35:18.411 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:18 smithi077 ceph-mon[32007]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:18 smithi092 ceph-mon[36049]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-27T08:35:18.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:18 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1111949371' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T08:35:18.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:18 smithi092 ceph-mon[36049]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:18 smithi179 ceph-mon[36481]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-27T08:35:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:18 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1111949371' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-27T08:35:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:18 smithi179 ceph-mon[36481]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 -- 172.21.15.77:0/2799355771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81041a0 msgr2=0x7fb5e81081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 --2- 172.21.15.77:0/2799355771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81041a0 0x7fb5e81081c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00b0d0 tx=0x7fb5dc00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:35:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 -- 172.21.15.77:0/2799355771 shutdown_connections 2024-05-27T08:35:19.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 --2- 172.21.15.77:0/2799355771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e81088f0 0x7fb5e8110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 --2- 172.21.15.77:0/2799355771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e81037f0 0x7fb5e8103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 --2- 172.21.15.77:0/2799355771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81041a0 0x7fb5e81081c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 -- 172.21.15.77:0/2799355771 >> 172.21.15.77:0/2799355771 conn(0x7fb5e807a6a0 msgr2=0x7fb5e807aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 -- 172.21.15.77:0/2799355771 shutdown_connections 2024-05-27T08:35:19.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 -- 172.21.15.77:0/2799355771 wait complete. 2024-05-27T08:35:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 Processor -- start 2024-05-27T08:35:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.623+0000 7fb5f0256700 1 -- start start 2024-05-27T08:35:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5f0256700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e81037f0 0x7fb5e819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5f0256700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e81041a0 0x7fb5e819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5f0256700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81088f0 0x7fb5e81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:19.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5f0256700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e81136b0 con 0x7fb5e81088f0 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5f0256700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8113530 con 0x7fb5e81041a0 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5f0256700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8113830 con 0x7fb5e81037f0 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81088f0 0x7fb5e81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81088f0 0x7fb5e81999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53700/0 (socket says 172.21.15.77:53700) 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 -- 172.21.15.77:0/3809142095 learned_addr learned my addr 172.21.15.77:0/3809142095 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5edff2700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e81037f0 0x7fb5e819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ed7f1700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e81041a0 0x7fb5e819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 -- 172.21.15.77:0/3809142095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e81037f0 msgr2=0x7fb5e819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e81037f0 0x7fb5e819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 -- 172.21.15.77:0/3809142095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e81041a0 msgr2=0x7fb5e819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:19.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e81041a0 0x7fb5e819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 -- 172.21.15.77:0/3809142095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e819a110 con 0x7fb5e81088f0 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ed7f1700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e81041a0 0x7fb5e819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5ee7f3700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81088f0 0x7fb5e81999b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800e940 tx=0x7fb5d800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5e6ffd700 1 -- 172.21.15.77:0/3809142095 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800cbd0 con 0x7fb5e81088f0 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5e6ffd700 1 -- 172.21.15.77:0/3809142095 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d800ee90 con 0x7fb5e81088f0 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e819a400 con 0x7fb5e81088f0 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.624+0000 7fb5e6ffd700 1 -- 172.21.15.77:0/3809142095 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8005590 con 0x7fb5e81088f0 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.625+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81a66c0 con 0x7fb5e81088f0 2024-05-27T08:35:19.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.626+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e806a230 con 0x7fb5e81088f0 2024-05-27T08:35:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.627+0000 7fb5e6ffd700 1 -- 172.21.15.77:0/3809142095 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb5d8004750 con 0x7fb5e81088f0 2024-05-27T08:35:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.627+0000 7fb5e6ffd700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb5d00b2880 0x7fb5d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.627+0000 7fb5edff2700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb5d00b2880 0x7fb5d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.628+0000 7fb5e6ffd700 1 -- 172.21.15.77:0/3809142095 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fb5d80e2300 con 0x7fb5e81088f0 2024-05-27T08:35:19.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.628+0000 7fb5edff2700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb5d00b2880 0x7fb5d00b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5e819aaf0 tx=0x7fb5e00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:19.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.632+0000 7fb5e6ffd700 1 -- 172.21.15.77:0/3809142095 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5d80aa170 con 0x7fb5e81088f0 2024-05-27T08:35:19.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:19.881+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fb5e802e560 con 0x7fb5e81088f0 2024-05-27T08:35:20.224 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.223+0000 7fb5e6ffd700 1 -- 172.21.15.77:0/3809142095 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fb5d80a9a70 con 0x7fb5e81088f0 2024-05-27T08:35:20.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.225+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb5d00b2880 msgr2=0x7fb5d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:20.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.225+0000 7fb5f0256700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb5d00b2880 0x7fb5d00b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5e819aaf0 tx=0x7fb5e00073d0 comp rx=0 tx=0).stop 2024-05-27T08:35:20.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.225+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81088f0 msgr2=0x7fb5e81999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:20.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.225+0000 7fb5f0256700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81088f0 0x7fb5e81999b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5d800e940 tx=0x7fb5d800ec50 comp rx=0 tx=0).stop 2024-05-27T08:35:20.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.226+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 shutdown_connections 2024-05-27T08:35:20.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.226+0000 7fb5f0256700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb5e81088f0 0x7fb5e81999b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:20.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.226+0000 7fb5f0256700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb5d00b2880 0x7fb5d00b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:20.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.226+0000 7fb5f0256700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e81037f0 0x7fb5e819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:20.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.226+0000 7fb5f0256700 1 --2- 172.21.15.77:0/3809142095 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5e81041a0 0x7fb5e819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:20.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.226+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 >> 172.21.15.77:0/3809142095 conn(0x7fb5e807a6a0 msgr2=0x7fb5e8101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:20.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.226+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 shutdown_connections 2024-05-27T08:35:20.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:20.226+0000 7fb5f0256700 1 -- 172.21.15.77:0/3809142095 wait complete. 2024-05-27T08:35:20.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:20 smithi092 ceph-mon[36049]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T08:35:20.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:20 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3809142095' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T08:35:20.547 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:20 smithi077 ceph-mon[32007]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T08:35:20.547 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:20 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3809142095' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T08:35:20.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:20 smithi179 ceph-mon[36481]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-27T08:35:20.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:20 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3809142095' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-27T08:35:20.835 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-27T08:35:21.076 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:21.458 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:21 smithi077 ceph-mon[32007]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T08:35:21.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:21 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3809142095' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T08:35:21.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:21 smithi077 ceph-mon[32007]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T08:35:21.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:21 smithi077 ceph-mon[32007]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:21.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:21 smithi077 ceph-mon[32007]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-27T08:35:21.459 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:21 smithi077 ceph-mon[32007]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-27T08:35:21.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:21 smithi092 ceph-mon[36049]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T08:35:21.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:21 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3809142095' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T08:35:21.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:21 smithi092 ceph-mon[36049]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T08:35:21.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:21 smithi092 ceph-mon[36049]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:21.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:21 smithi092 ceph-mon[36049]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-27T08:35:21.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:21 smithi092 ceph-mon[36049]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-27T08:35:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:21 smithi179 ceph-mon[36481]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-27T08:35:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:21 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3809142095' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-27T08:35:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:21 smithi179 ceph-mon[36481]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-27T08:35:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:21 smithi179 ceph-mon[36481]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-27T08:35:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:21 smithi179 ceph-mon[36481]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-05-27T08:35:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:21 smithi179 ceph-mon[36481]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-05-27T08:35:22.415 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:22 smithi077 ceph-mon[32007]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-27T08:35:22.415 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:22 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] up:active 2024-05-27T08:35:22.415 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:22 smithi077 ceph-mon[32007]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T08:35:22.415 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:22 smithi077 ceph-mon[32007]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-27T08:35:22.416 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:22 smithi077 ceph-mon[32007]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-27T08:35:22.416 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:22 smithi077 ceph-mon[32007]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-27T08:35:22.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:22 smithi179 ceph-mon[36481]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-27T08:35:22.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:22 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] up:active 2024-05-27T08:35:22.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:22 smithi179 ceph-mon[36481]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T08:35:22.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:22 smithi179 ceph-mon[36481]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-27T08:35:22.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:22 smithi179 ceph-mon[36481]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-27T08:35:22.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:22 smithi179 ceph-mon[36481]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-27T08:35:22.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 -- 172.21.15.77:0/76763575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901088f0 msgr2=0x7f3090110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:22.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 --2- 172.21.15.77:0/76763575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901088f0 0x7f3090110470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f30880099c0 tx=0x7f3088009cd0 comp rx=0 tx=0).stop 2024-05-27T08:35:22.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 -- 172.21.15.77:0/76763575 shutdown_connections 2024-05-27T08:35:22.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 --2- 172.21.15.77:0/76763575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901088f0 0x7f3090110470 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 --2- 172.21.15.77:0/76763575 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30901037f0 0x7f3090103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 --2- 172.21.15.77:0/76763575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30901041a0 0x7f30901081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 -- 172.21.15.77:0/76763575 >> 172.21.15.77:0/76763575 conn(0x7f309007a6a0 msgr2=0x7f309007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 -- 172.21.15.77:0/76763575 shutdown_connections 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.628+0000 7f3095b1a700 1 -- 172.21.15.77:0/76763575 wait complete. 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f3095b1a700 1 Processor -- start 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f3095b1a700 1 -- start start 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f3095b1a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901037f0 0x7f309019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f3095b1a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30901041a0 0x7f309019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f3095b1a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30901088f0 0x7f3090199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f3095b1a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30901136e0 con 0x7f30901037f0 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f3095b1a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3090113560 con 0x7f30901088f0 2024-05-27T08:35:22.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f3095b1a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3090113860 con 0x7f30901041a0 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901037f0 0x7f309019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901037f0 0x7f309019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53716/0 (socket says 172.21.15.77:53716) 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308f7fe700 1 -- 172.21.15.77:0/2812144774 learned_addr learned my addr 172.21.15.77:0/2812144774 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308effd700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30901041a0 0x7f309019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308ffff700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30901088f0 0x7f3090199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308f7fe700 1 -- 172.21.15.77:0/2812144774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30901041a0 msgr2=0x7f309019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308f7fe700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30901041a0 0x7f309019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308f7fe700 1 -- 172.21.15.77:0/2812144774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30901088f0 msgr2=0x7f3090199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308f7fe700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30901088f0 0x7f3090199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308f7fe700 1 -- 172.21.15.77:0/2812144774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f309019a0c0 con 0x7f30901037f0 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.629+0000 7f308ffff700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30901088f0 0x7f3090199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.630+0000 7f308f7fe700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901037f0 0x7f309019f7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f308000ada0 tx=0x7f308000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:22.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.630+0000 7f308cff9700 1 -- 172.21.15.77:0/2812144774 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080017020 con 0x7f30901037f0 2024-05-27T08:35:22.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.630+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f309019a3b0 con 0x7f30901037f0 2024-05-27T08:35:22.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.630+0000 7f308cff9700 1 -- 172.21.15.77:0/2812144774 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f308000ce30 con 0x7f30901037f0 2024-05-27T08:35:22.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.630+0000 7f308cff9700 1 -- 172.21.15.77:0/2812144774 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080005480 con 0x7f30901037f0 2024-05-27T08:35:22.634 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.630+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30901a6670 con 0x7f30901037f0 2024-05-27T08:35:22.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.631+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f309006a230 con 0x7f30901037f0 2024-05-27T08:35:22.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.632+0000 7f308cff9700 1 -- 172.21.15.77:0/2812144774 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3080004ad0 con 0x7f30901037f0 2024-05-27T08:35:22.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.633+0000 7f308cff9700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30780b2880 0x7f30780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:22.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.633+0000 7f308effd700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30780b2880 0x7f30780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:22.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.633+0000 7f308cff9700 1 -- 172.21.15.77:0/2812144774 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f30800e1ee0 con 0x7f30901037f0 2024-05-27T08:35:22.635 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.633+0000 7f308effd700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30780b2880 0x7f30780b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3090108170 tx=0x7f308400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:22.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.636+0000 7f308cff9700 1 -- 172.21.15.77:0/2812144774 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3080021020 con 0x7f30901037f0 2024-05-27T08:35:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:22 smithi092 ceph-mon[36049]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-27T08:35:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:22 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] up:active 2024-05-27T08:35:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:22 smithi092 ceph-mon[36049]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-27T08:35:22.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:22 smithi092 ceph-mon[36049]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-05-27T08:35:22.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:22 smithi092 ceph-mon[36049]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-05-27T08:35:22.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:22 smithi092 ceph-mon[36049]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-05-27T08:35:22.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:22.891+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f309019ae90 con 0x7f30901037f0 2024-05-27T08:35:23.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.256+0000 7f308cff9700 1 -- 172.21.15.77:0/2812144774 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f30800a9de0 con 0x7f30901037f0 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30780b2880 msgr2=0x7f30780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30780b2880 0x7f30780b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3090108170 tx=0x7f308400b3c0 comp rx=0 tx=0).stop 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901037f0 msgr2=0x7f309019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901037f0 0x7f309019f7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f308000ada0 tx=0x7f308000c330 comp rx=0 tx=0).stop 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 shutdown_connections 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30901088f0 0x7f3090199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30901037f0 0x7f309019f7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30780b2880 0x7f30780b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:23.260 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 --2- 172.21.15.77:0/2812144774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30901041a0 0x7f309019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.258+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 >> 172.21.15.77:0/2812144774 conn(0x7f309007a6a0 msgr2=0x7f30901015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.259+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 shutdown_connections 2024-05-27T08:35:23.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:23.259+0000 7f3095b1a700 1 -- 172.21.15.77:0/2812144774 wait complete. 2024-05-27T08:35:23.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:23 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] up:active 2024-05-27T08:35:23.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:23 smithi077 ceph-mon[32007]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T08:35:23.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:23 smithi077 ceph-mon[32007]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-27T08:35:23.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:23 smithi077 ceph-mon[32007]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-27T08:35:23.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:23 smithi077 ceph-mon[32007]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-27T08:35:23.554 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:23 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2812144774' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T08:35:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:23 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] up:active 2024-05-27T08:35:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:23 smithi179 ceph-mon[36481]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T08:35:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:23 smithi179 ceph-mon[36481]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-27T08:35:23.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:23 smithi179 ceph-mon[36481]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-27T08:35:23.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:23 smithi179 ceph-mon[36481]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-27T08:35:23.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:23 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2812144774' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T08:35:23.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:23 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] up:active 2024-05-27T08:35:23.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:23 smithi092 ceph-mon[36049]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-27T08:35:23.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:23 smithi092 ceph-mon[36049]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-05-27T08:35:23.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:23 smithi092 ceph-mon[36049]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-05-27T08:35:23.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:23 smithi092 ceph-mon[36049]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-05-27T08:35:23.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:23 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2812144774' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-27T08:35:23.949 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-27T08:35:24.191 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:24.486 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: 2024-05-27T08:35:24.224+0000 7f2d3a3fc700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-27T08:35:24.486 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: 2024-05-27T08:35:24.224+0000 7f2d3a3fc700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-27T08:35:24.486 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: 2024-05-27T08:35:24.224+0000 7f2d3a3fc700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2812144774' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] up:active 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: Cluster is now healthy 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T08:35:24.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:24 smithi092 ceph-mon[36049]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-27T08:35:24.556 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-05-27T08:35:24.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2812144774' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T08:35:24.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] up:active 2024-05-27T08:35:24.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T08:35:24.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T08:35:24.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: Cluster is now healthy 2024-05-27T08:35:24.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-27T08:35:24.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T08:35:24.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:24 smithi077 ceph-mon[32007]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-27T08:35:24.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-05-27T08:35:24.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2812144774' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-27T08:35:24.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] up:active 2024-05-27T08:35:24.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-27T08:35:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-27T08:35:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: Cluster is now healthy 2024-05-27T08:35:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-05-27T08:35:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-27T08:35:24.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:24 smithi179 ceph-mon[36481]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-05-27T08:35:25.564 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:25 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] up:active 2024-05-27T08:35:25.565 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:25 smithi077 ceph-mon[32007]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T08:35:25.565 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:25 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:35:25.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:25 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] up:active 2024-05-27T08:35:25.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:25 smithi179 ceph-mon[36481]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T08:35:25.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:25 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:35:25.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:25 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] up:active 2024-05-27T08:35:25.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:25 smithi092 ceph-mon[36049]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-27T08:35:25.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:25 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:35:25.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 -- 172.21.15.77:0/1094554451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618103f90 msgr2=0x7f7618110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 --2- 172.21.15.77:0/1094554451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618103f90 0x7f7618110440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76080099c0 tx=0x7f7608009cd0 comp rx=0 tx=0).stop 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 -- 172.21.15.77:0/1094554451 shutdown_connections 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 --2- 172.21.15.77:0/1094554451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618103f90 0x7f7618110440 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 --2- 172.21.15.77:0/1094554451 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7618109670 0x7f7618109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 --2- 172.21.15.77:0/1094554451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76181035d0 0x7f7618103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 -- 172.21.15.77:0/1094554451 >> 172.21.15.77:0/1094554451 conn(0x7f761807a730 msgr2=0x7f761807ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 -- 172.21.15.77:0/1094554451 shutdown_connections 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 -- 172.21.15.77:0/1094554451 wait complete. 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.870+0000 7f761d215700 1 Processor -- start 2024-05-27T08:35:25.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761d215700 1 -- start start 2024-05-27T08:35:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761d215700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76181035d0 0x7f761819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761d215700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618103f90 0x7f761819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761d215700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618109670 0x7f7618199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761d215700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76181137d0 con 0x7f7618109670 2024-05-27T08:35:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761d215700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7618113650 con 0x7f76181035d0 2024-05-27T08:35:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761d215700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7618113950 con 0x7f7618103f90 2024-05-27T08:35:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618109670 0x7f7618199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:25.874 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618109670 0x7f7618199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53722/0 (socket says 172.21.15.77:53722) 2024-05-27T08:35:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 -- 172.21.15.77:0/1830208804 learned_addr learned my addr 172.21.15.77:0/1830208804 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f7616d9d700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76181035d0 0x7f761819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761659c700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618103f90 0x7f761819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 -- 172.21.15.77:0/1830208804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618103f90 msgr2=0x7f761819fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618103f90 0x7f761819fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 -- 172.21.15.77:0/1830208804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76181035d0 msgr2=0x7f761819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76181035d0 0x7f761819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:25.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 -- 172.21.15.77:0/1830208804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761819a270 con 0x7f7618109670 2024-05-27T08:35:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.871+0000 7f761759e700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618109670 0x7f7618199b10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76080059d0 tx=0x7f76080045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.872+0000 7f760ffff700 1 -- 172.21.15.77:0/1830208804 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608026070 con 0x7f7618109670 2024-05-27T08:35:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.872+0000 7f760ffff700 1 -- 172.21.15.77:0/1830208804 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760800f460 con 0x7f7618109670 2024-05-27T08:35:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.872+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761819a500 con 0x7f7618109670 2024-05-27T08:35:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.872+0000 7f760ffff700 1 -- 172.21.15.77:0/1830208804 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608009e60 con 0x7f7618109670 2024-05-27T08:35:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.872+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76181a67d0 con 0x7f7618109670 2024-05-27T08:35:25.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.873+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761810bb00 con 0x7f7618109670 2024-05-27T08:35:25.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.874+0000 7f760ffff700 1 -- 172.21.15.77:0/1830208804 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7608029070 con 0x7f7618109670 2024-05-27T08:35:25.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.875+0000 7f760ffff700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:25.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.875+0000 7f7616d9d700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:25.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.875+0000 7f760ffff700 1 -- 172.21.15.77:0/1830208804 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f76080e2c20 con 0x7f7618109670 2024-05-27T08:35:25.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.875+0000 7f7616d9d700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7618101c60 tx=0x7f761000d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:25.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:25.878+0000 7f760ffff700 1 -- 172.21.15.77:0/1830208804 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7608013070 con 0x7f7618109670 2024-05-27T08:35:26.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.133+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f761819aff0 con 0x7f7618109670 2024-05-27T08:35:26.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.273+0000 7f760ffff700 1 -- 172.21.15.77:0/1830208804 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f76080aab20 con 0x7f7618109670 2024-05-27T08:35:26.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.275+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 msgr2=0x7f76040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:26.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.275+0000 7f761d215700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7618101c60 tx=0x7f761000d040 comp rx=0 tx=0).stop 2024-05-27T08:35:26.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.275+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618109670 msgr2=0x7f7618199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:26.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.275+0000 7f761d215700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618109670 0x7f7618199b10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76080059d0 tx=0x7f76080045e0 comp rx=0 tx=0).stop 2024-05-27T08:35:26.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.275+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 shutdown_connections 2024-05-27T08:35:26.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.275+0000 7f761d215700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618109670 0x7f7618199b10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:26.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.276+0000 7f761d215700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f76181035d0 0x7f761819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:26.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.276+0000 7f761d215700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:26.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.276+0000 7f761d215700 1 --2- 172.21.15.77:0/1830208804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618103f90 0x7f761819fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:26.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.276+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 >> 172.21.15.77:0/1830208804 conn(0x7f761807a730 msgr2=0x7f76181021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:26.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.276+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 shutdown_connections 2024-05-27T08:35:26.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:26.276+0000 7f761d215700 1 -- 172.21.15.77:0/1830208804 wait complete. 2024-05-27T08:35:26.560 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:26 smithi077 ceph-mon[32007]: pgmap v297: 97 pgs: 97 active+clean; 583 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.8 KiB/s wr, 13 op/s 2024-05-27T08:35:26.560 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:26 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1830208804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T08:35:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:26 smithi179 ceph-mon[36481]: pgmap v297: 97 pgs: 97 active+clean; 583 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.8 KiB/s wr, 13 op/s 2024-05-27T08:35:26.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:26 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1830208804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T08:35:26.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:26 smithi092 ceph-mon[36049]: pgmap v297: 97 pgs: 97 active+clean; 583 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 3.8 KiB/s wr, 13 op/s 2024-05-27T08:35:26.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:26 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1830208804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-27T08:35:26.907 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-27T08:35:26.907 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-27T08:35:27.150 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:27.606 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:27 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1830208804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T08:35:27.606 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:27 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:35:27.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:27 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1830208804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T08:35:27.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:27 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:35:27.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:27 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1830208804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-27T08:35:27.761 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:27 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:35:28.528 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:28 smithi077 ceph-mon[32007]: pgmap v298: 97 pgs: 97 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-27T08:35:28.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:28 smithi179 ceph-mon[36481]: pgmap v298: 97 pgs: 97 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-27T08:35:28.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:28 smithi092 ceph-mon[36049]: pgmap v298: 97 pgs: 97 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 3.7 KiB/s wr, 13 op/s 2024-05-27T08:35:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 -- 172.21.15.77:0/1184531398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 msgr2=0x7f66241103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 --2- 172.21.15.77:0/1184531398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 0x7f66241103d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f661c00b0a0 tx=0x7f661c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:35:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 -- 172.21.15.77:0/1184531398 shutdown_connections 2024-05-27T08:35:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 --2- 172.21.15.77:0/1184531398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 0x7f66241103d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 --2- 172.21.15.77:0/1184531398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66241095b0 0x7f6624109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 --2- 172.21.15.77:0/1184531398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662406bc70 0x7f662406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 -- 172.21.15.77:0/1184531398 >> 172.21.15.77:0/1184531398 conn(0x7f66240793f0 msgr2=0x7f6624079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 -- 172.21.15.77:0/1184531398 shutdown_connections 2024-05-27T08:35:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.759+0000 7f662c5ab700 1 -- 172.21.15.77:0/1184531398 wait complete. 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662c5ab700 1 Processor -- start 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662c5ab700 1 -- start start 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662c5ab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662406bc70 0x7f6624102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662c5ab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 0x7f6624102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662c5ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66241095b0 0x7f6624102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662c5ab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6624113750 con 0x7f662406c630 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662c5ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f66241135d0 con 0x7f66241095b0 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662c5ab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66241138d0 con 0x7f662406bc70 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f6629b46700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 0x7f6624102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f6629b46700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 0x7f6624102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53746/0 (socket says 172.21.15.77:53746) 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f6629b46700 1 -- 172.21.15.77:0/3889230432 learned_addr learned my addr 172.21.15.77:0/3889230432 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662a347700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662406bc70 0x7f6624102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f662ab48700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66241095b0 0x7f6624102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:28.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f6629b46700 1 -- 172.21.15.77:0/3889230432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662406bc70 msgr2=0x7f6624102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f6629b46700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662406bc70 0x7f6624102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f6629b46700 1 -- 172.21.15.77:0/3889230432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66241095b0 msgr2=0x7f6624102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f6629b46700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66241095b0 0x7f6624102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.760+0000 7f6629b46700 1 -- 172.21.15.77:0/3889230432 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66241a8120 con 0x7f662406c630 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.761+0000 7f6629b46700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 0x7f6624102a80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f661400b530 tx=0x7f661400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.761+0000 7f661b7fe700 1 -- 172.21.15.77:0/3889230432 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614015020 con 0x7f662406c630 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.761+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66241a8410 con 0x7f662406c630 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.761+0000 7f661b7fe700 1 -- 172.21.15.77:0/3889230432 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6614011ce0 con 0x7f662406c630 2024-05-27T08:35:28.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.761+0000 7f661b7fe700 1 -- 172.21.15.77:0/3889230432 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614010dd0 con 0x7f662406c630 2024-05-27T08:35:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.761+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66241a8950 con 0x7f662406c630 2024-05-27T08:35:28.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.762+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662406a230 con 0x7f662406c630 2024-05-27T08:35:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.763+0000 7f661b7fe700 1 -- 172.21.15.77:0/3889230432 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6614018020 con 0x7f662406c630 2024-05-27T08:35:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.764+0000 7f661b7fe700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f66100b2880 0x7f66100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.764+0000 7f662a347700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f66100b2880 0x7f66100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.764+0000 7f661b7fe700 1 -- 172.21.15.77:0/3889230432 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f66140e1d80 con 0x7f662406c630 2024-05-27T08:35:28.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.764+0000 7f662a347700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f66100b2880 0x7f66100b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6620005950 tx=0x7f662000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:28.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:28.767+0000 7f661b7fe700 1 -- 172.21.15.77:0/3889230432 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f661401f020 con 0x7f662406c630 2024-05-27T08:35:29.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.006+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6624063980 con 0x7f66100b2880 2024-05-27T08:35:29.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.037+0000 7f661b7fe700 1 -- 172.21.15.77:0/3889230432 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6624063980 con 0x7f66100b2880 2024-05-27T08:35:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f66100b2880 msgr2=0x7f66100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:29.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f66100b2880 0x7f66100b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6620005950 tx=0x7f662000b410 comp rx=0 tx=0).stop 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 msgr2=0x7f6624102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 0x7f6624102a80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f661400b530 tx=0x7f661400b840 comp rx=0 tx=0).stop 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 shutdown_connections 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f66241095b0 0x7f6624102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f66100b2880 0x7f66100b4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f662406bc70 0x7f6624102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 --2- 172.21.15.77:0/3889230432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f662406c630 0x7f6624102a80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 >> 172.21.15.77:0/3889230432 conn(0x7f66240793f0 msgr2=0x7f66241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:29.041 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 shutdown_connections 2024-05-27T08:35:29.042 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:29.039+0000 7f662c5ab700 1 -- 172.21.15.77:0/3889230432 wait complete. 2024-05-27T08:35:29.292 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:35:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:35:29.011+0000 7f7217219700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:35:29.292 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:35:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:35:29.011+0000 7f7217219700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:35:29.292 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:35:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:35:29.011+0000 7f7217219700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:35:29.292 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:35:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:35:29.011+0000 7f7217219700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:35:29.292 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:35:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:35:29.011+0000 7f7217219700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:35:29.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:29 smithi179 ceph-mon[36481]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:29.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:29 smithi077 ceph-mon[32007]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:29.702 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-27T08:35:29.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:29 smithi092 ceph-mon[36049]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:29.941 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:30.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:30 smithi077 ceph-mon[32007]: pgmap v299: 97 pgs: 97 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-27T08:35:30.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:30 smithi077 ceph-mon[32007]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-05-27T08:35:30.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:30 smithi179 ceph-mon[36481]: pgmap v299: 97 pgs: 97 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-27T08:35:30.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:30 smithi179 ceph-mon[36481]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-05-27T08:35:30.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:30 smithi092 ceph-mon[36049]: pgmap v299: 97 pgs: 97 active+clean; 584 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-05-27T08:35:30.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:30 smithi092 ceph-mon[36049]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 -- 172.21.15.77:0/3820551404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c01095b0 msgr2=0x7fe3c0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3820551404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c01095b0 0x7fe3c0109990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009990 tx=0x7fe3b0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 -- 172.21.15.77:0/3820551404 shutdown_connections 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3820551404 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3c006bbe0 0x7fe3c01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3820551404 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c01095b0 0x7fe3c0109990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3820551404 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c006b220 0x7fe3c006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 -- 172.21.15.77:0/3820551404 >> 172.21.15.77:0/3820551404 conn(0x7fe3c00793f0 msgr2=0x7fe3c0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 -- 172.21.15.77:0/3820551404 shutdown_connections 2024-05-27T08:35:31.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.607+0000 7fe3c6394700 1 -- 172.21.15.77:0/3820551404 wait complete. 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c6394700 1 Processor -- start 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c6394700 1 -- start start 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c6394700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c006b220 0x7fe3c0102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c6394700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c006bbe0 0x7fe3c0102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c6394700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3c01095b0 0x7fe3c0102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c6394700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0113740 con 0x7fe3c006b220 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c6394700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01135c0 con 0x7fe3c006bbe0 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c6394700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c01138c0 con 0x7fe3c01095b0 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c006b220 0x7fe3c0102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:31.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3bffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c006b220 0x7fe3c0102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37452/0 (socket says 172.21.15.77:37452) 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3bffff700 1 -- 172.21.15.77:0/3301249274 learned_addr learned my addr 172.21.15.77:0/3301249274 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3b7fff700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c006bbe0 0x7fe3c0102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.608+0000 7fe3c4931700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3c01095b0 0x7fe3c0102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bffff700 1 -- 172.21.15.77:0/3301249274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3c01095b0 msgr2=0x7fe3c0102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bffff700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3c01095b0 0x7fe3c0102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bffff700 1 -- 172.21.15.77:0/3301249274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c006bbe0 msgr2=0x7fe3c0102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bffff700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c006bbe0 0x7fe3c0102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bffff700 1 -- 172.21.15.77:0/3301249274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c01a8180 con 0x7fe3c006b220 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3b7fff700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c006bbe0 0x7fe3c0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bffff700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c006b220 0x7fe3c0102510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000b5c0 tx=0x7fe3b0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bdffb700 1 -- 172.21.15.77:0/3301249274 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0026070 con 0x7fe3c006b220 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bdffb700 1 -- 172.21.15.77:0/3301249274 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3b0004170 con 0x7fe3c006b220 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3bdffb700 1 -- 172.21.15.77:0/3301249274 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0005250 con 0x7fe3c006b220 2024-05-27T08:35:31.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c01a8410 con 0x7fe3c006b220 2024-05-27T08:35:31.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.609+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c01a88c0 con 0x7fe3c006b220 2024-05-27T08:35:31.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.610+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c010ba90 con 0x7fe3c006b220 2024-05-27T08:35:31.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.611+0000 7fe3bdffb700 1 -- 172.21.15.77:0/3301249274 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe3b0029070 con 0x7fe3c006b220 2024-05-27T08:35:31.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.612+0000 7fe3bdffb700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe3ac0b2990 0x7fe3ac0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:31.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.612+0000 7fe3b7fff700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe3ac0b2990 0x7fe3ac0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:31.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.612+0000 7fe3bdffb700 1 -- 172.21.15.77:0/3301249274 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fe3b00e2e10 con 0x7fe3c006b220 2024-05-27T08:35:31.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.612+0000 7fe3b7fff700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe3ac0b2990 0x7fe3ac0b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0079ad0 tx=0x7fe3a8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:31.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.616+0000 7fe3bdffb700 1 -- 172.21.15.77:0/3301249274 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3b0013070 con 0x7fe3c006b220 2024-05-27T08:35:31.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.852+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe3c0063980 con 0x7fe3ac0b2990 2024-05-27T08:35:31.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.880+0000 7fe3bdffb700 1 -- 172.21.15.77:0/3301249274 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe3c0063980 con 0x7fe3ac0b2990 2024-05-27T08:35:31.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.882+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe3ac0b2990 msgr2=0x7fe3ac0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:31.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.882+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe3ac0b2990 0x7fe3ac0b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0079ad0 tx=0x7fe3a8005f50 comp rx=0 tx=0).stop 2024-05-27T08:35:31.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c006b220 msgr2=0x7fe3c0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:31.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c006b220 0x7fe3c0102510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000b5c0 tx=0x7fe3b0004630 comp rx=0 tx=0).stop 2024-05-27T08:35:31.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 shutdown_connections 2024-05-27T08:35:31.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe3ac0b2990 0x7fe3ac0b4e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3c01095b0 0x7fe3c0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3c006b220 0x7fe3c0102510 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 --2- 172.21.15.77:0/3301249274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3c006bbe0 0x7fe3c0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:31.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 >> 172.21.15.77:0/3301249274 conn(0x7fe3c00793f0 msgr2=0x7fe3c01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:31.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 shutdown_connections 2024-05-27T08:35:31.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:31.883+0000 7fe3c6394700 1 -- 172.21.15.77:0/3301249274 wait complete. 2024-05-27T08:35:32.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:32 smithi179 ceph-mon[36481]: pgmap v300: 97 pgs: 97 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-05-27T08:35:32.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:32 smithi179 ceph-mon[36481]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:32.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:32 smithi077 ceph-mon[32007]: pgmap v300: 97 pgs: 97 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-05-27T08:35:32.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:32 smithi077 ceph-mon[32007]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:32.681 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-27T08:35:32.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:32 smithi092 ceph-mon[36049]: pgmap v300: 97 pgs: 97 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-05-27T08:35:32.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:32 smithi092 ceph-mon[36049]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:32.926 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:34.347 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:34 smithi077 ceph-mon[32007]: pgmap v301: 97 pgs: 97 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-05-27T08:35:34.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 -- 172.21.15.77:0/1330215643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 msgr2=0x7f397c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:34.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 --2- 172.21.15.77:0/1330215643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 0x7f397c108890 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f396c009990 tx=0x7f396c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 -- 172.21.15.77:0/1330215643 shutdown_connections 2024-05-27T08:35:34.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 --2- 172.21.15.77:0/1330215643 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102e30 0x7f397c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 --2- 172.21.15.77:0/1330215643 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 0x7f397c108890 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 --2- 172.21.15.77:0/1330215643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f397c102470 0x7f397c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 -- 172.21.15.77:0/1330215643 >> 172.21.15.77:0/1330215643 conn(0x7f397c079410 msgr2=0x7f397c079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:34.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 -- 172.21.15.77:0/1330215643 shutdown_connections 2024-05-27T08:35:34.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.448+0000 7f39820cf700 1 -- 172.21.15.77:0/1330215643 wait complete. 2024-05-27T08:35:34.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f39820cf700 1 Processor -- start 2024-05-27T08:35:34.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f39820cf700 1 -- start start 2024-05-27T08:35:34.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f39820cf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f397c102470 0x7f397c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:34.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f39820cf700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102e30 0x7f397c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:34.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f39820cf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 0x7f397c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:34.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f39820cf700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f397c112710 con 0x7f397c1084b0 2024-05-27T08:35:34.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f39820cf700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f397c112590 con 0x7f397c102e30 2024-05-27T08:35:34.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f39820cf700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f397c112890 con 0x7f397c102470 2024-05-27T08:35:34.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.449+0000 7f397bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 0x7f397c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:34.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 0x7f397c104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37480/0 (socket says 172.21.15.77:37480) 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397bfff700 1 -- 172.21.15.77:0/4164859392 learned_addr learned my addr 172.21.15.77:0/4164859392 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397affd700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102e30 0x7f397c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397b7fe700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f397c102470 0x7f397c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397bfff700 1 -- 172.21.15.77:0/4164859392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f397c102470 msgr2=0x7f397c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397bfff700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f397c102470 0x7f397c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397bfff700 1 -- 172.21.15.77:0/4164859392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102e30 msgr2=0x7f397c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397bfff700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102e30 0x7f397c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397bfff700 1 -- 172.21.15.77:0/4164859392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c1a8360 con 0x7f397c1084b0 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f397bfff700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 0x7f397c104b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f397400b970 tx=0x7f397400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f3978ff9700 1 -- 172.21.15.77:0/4164859392 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974019070 con 0x7f397c1084b0 2024-05-27T08:35:34.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f3978ff9700 1 -- 172.21.15.77:0/4164859392 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f397400cde0 con 0x7f397c1084b0 2024-05-27T08:35:34.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397c1a8650 con 0x7f397c1084b0 2024-05-27T08:35:34.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f3978ff9700 1 -- 172.21.15.77:0/4164859392 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974007370 con 0x7f397c1084b0 2024-05-27T08:35:34.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.450+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397c1a8b90 con 0x7f397c1084b0 2024-05-27T08:35:34.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.451+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f397c06a230 con 0x7f397c1084b0 2024-05-27T08:35:34.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.453+0000 7f3978ff9700 1 -- 172.21.15.77:0/4164859392 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f397401c070 con 0x7f397c1084b0 2024-05-27T08:35:34.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.453+0000 7f3978ff9700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39680b2990 0x7f39680b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:34.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.453+0000 7f397b7fe700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39680b2990 0x7f39680b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:34.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.453+0000 7f3978ff9700 1 -- 172.21.15.77:0/4164859392 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f39740e1ca0 con 0x7f397c1084b0 2024-05-27T08:35:34.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.454+0000 7f397b7fe700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39680b2990 0x7f39680b4e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f396c009750 tx=0x7f396c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:34.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.457+0000 7f3978ff9700 1 -- 172.21.15.77:0/4164859392 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3974023070 con 0x7f397c1084b0 2024-05-27T08:35:34.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:34 smithi179 ceph-mon[36481]: pgmap v301: 97 pgs: 97 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-05-27T08:35:34.699 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.697+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f397c063980 con 0x7f39680b2990 2024-05-27T08:35:34.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.702+0000 7f3978ff9700 1 -- 172.21.15.77:0/4164859392 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f397c063980 con 0x7f39680b2990 2024-05-27T08:35:34.704 INFO:teuthology.orchestra.run.smithi077.stdout:/volumes/qa/sv_0/0ce4d4cb-afea-465f-97a5-471cca9d29f1 2024-05-27T08:35:34.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39680b2990 msgr2=0x7f39680b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:34.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39680b2990 0x7f39680b4e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f396c009750 tx=0x7f396c01a040 comp rx=0 tx=0).stop 2024-05-27T08:35:34.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 msgr2=0x7f397c104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:34.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 0x7f397c104b00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f397400b970 tx=0x7f397400bc80 comp rx=0 tx=0).stop 2024-05-27T08:35:34.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 shutdown_connections 2024-05-27T08:35:34.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f397c1084b0 0x7f397c104b00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f39680b2990 0x7f39680b4e50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f397c102470 0x7f397c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 --2- 172.21.15.77:0/4164859392 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f397c102e30 0x7f397c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:34.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 >> 172.21.15.77:0/4164859392 conn(0x7f397c079410 msgr2=0x7f397c06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:34.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 shutdown_connections 2024-05-27T08:35:34.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:34.705+0000 7f39820cf700 1 -- 172.21.15.77:0/4164859392 wait complete. 2024-05-27T08:35:34.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:34 smithi092 ceph-mon[36049]: pgmap v301: 97 pgs: 97 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-05-27T08:35:35.441 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-27T08:35:35.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:35 smithi179 ceph-mon[36481]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:35.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:35 smithi077 ceph-mon[32007]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:35.681 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:35.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:35 smithi092 ceph-mon[36049]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:36.579 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:36 smithi077 ceph-mon[32007]: pgmap v302: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-05-27T08:35:36.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:36 smithi179 ceph-mon[36481]: pgmap v302: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-05-27T08:35:36.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:36 smithi092 ceph-mon[36049]: pgmap v302: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-05-27T08:35:37.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/2622573478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff01095b0 msgr2=0x7f8ff0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:37.081 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/2622573478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff01095b0 0x7f8ff0109990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009990 tx=0x7f8fe4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/2622573478 shutdown_connections 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/2622573478 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff006c650 0x7f8ff01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/2622573478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff01095b0 0x7f8ff0109990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/2622573478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ff006bc90 0x7f8ff006c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/2622573478 >> 172.21.15.77:0/2622573478 conn(0x7f8ff0079410 msgr2=0x7f8ff0079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/2622573478 shutdown_connections 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.078+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/2622573478 wait complete. 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff6cdc700 1 Processor -- start 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff6cdc700 1 -- start start 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff6cdc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff006bc90 0x7f8ff0199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:37.082 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff6cdc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ff006c650 0x7f8ff019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff6cdc700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01095b0 0x7f8ff019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff6cdc700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff01137c0 con 0x7f8ff006bc90 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff6cdc700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0113640 con 0x7f8ff01095b0 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff6cdc700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0113940 con 0x7f8ff006c650 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff4a78700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff006bc90 0x7f8ff0199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff4a78700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff006bc90 0x7f8ff0199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37504/0 (socket says 172.21.15.77:37504) 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff4a78700 1 -- 172.21.15.77:0/624288438 learned_addr learned my addr 172.21.15.77:0/624288438 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8feffff700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ff006c650 0x7f8ff019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff4a78700 1 -- 172.21.15.77:0/624288438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ff006c650 msgr2=0x7f8ff019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.079+0000 7f8ff4a78700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ff006c650 0x7f8ff019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8ff4a78700 1 -- 172.21.15.77:0/624288438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01095b0 msgr2=0x7f8ff019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8ff4a78700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01095b0 0x7f8ff019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8ff4a78700 1 -- 172.21.15.77:0/624288438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff019eda0 con 0x7f8ff006bc90 2024-05-27T08:35:37.083 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8ff4a78700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff006bc90 0x7f8ff0199d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4000c00 tx=0x7f8fe40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8fedffb700 1 -- 172.21.15.77:0/624288438 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4027070 con 0x7f8ff006bc90 2024-05-27T08:35:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff019f030 con 0x7f8ff006bc90 2024-05-27T08:35:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8fedffb700 1 -- 172.21.15.77:0/624288438 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe40041d0 con 0x7f8ff006bc90 2024-05-27T08:35:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8fedffb700 1 -- 172.21.15.77:0/624288438 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400f460 con 0x7f8ff006bc90 2024-05-27T08:35:37.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.080+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01a6910 con 0x7f8ff006bc90 2024-05-27T08:35:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.081+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff010ba90 con 0x7f8ff006bc90 2024-05-27T08:35:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.082+0000 7f8fedffb700 1 -- 172.21.15.77:0/624288438 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8fe402a070 con 0x7f8ff006bc90 2024-05-27T08:35:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.083+0000 7f8fedffb700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8fe00b2910 0x7f8fe00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.083+0000 7f8feffff700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8fe00b2910 0x7f8fe00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.083+0000 7f8fedffb700 1 -- 172.21.15.77:0/624288438 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f8fe40e3630 con 0x7f8ff006bc90 2024-05-27T08:35:37.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.083+0000 7f8feffff700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8fe00b2910 0x7f8fe00b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006fd0 tx=0x7f8fdc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:37.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.086+0000 7f8fedffb700 1 -- 172.21.15.77:0/624288438 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8fe4031070 con 0x7f8ff006bc90 2024-05-27T08:35:37.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.327+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8ff0063980 con 0x7f8fe00b2910 2024-05-27T08:35:37.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.358+0000 7f8fedffb700 1 -- 172.21.15.77:0/624288438 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ff0063980 con 0x7f8fe00b2910 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8fe00b2910 msgr2=0x7f8fe00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8fe00b2910 0x7f8fe00b4dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc006fd0 tx=0x7f8fdc008040 comp rx=0 tx=0).stop 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff006bc90 msgr2=0x7f8ff0199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff006bc90 0x7f8ff0199d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4000c00 tx=0x7f8fe40047b0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 shutdown_connections 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ff01095b0 0x7f8ff019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8ff006bc90 0x7f8ff0199d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8fe00b2910 0x7f8fe00b4dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 --2- 172.21.15.77:0/624288438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ff006c650 0x7f8ff019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.360+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 >> 172.21.15.77:0/624288438 conn(0x7f8ff0079410 msgr2=0x7f8ff0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:37.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.361+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 shutdown_connections 2024-05-27T08:35:37.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:37.361+0000 7f8ff6cdc700 1 -- 172.21.15.77:0/624288438 wait complete. 2024-05-27T08:35:38.057 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-27T08:35:38.299 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:38.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:38 smithi179 ceph-mon[36481]: pgmap v303: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-05-27T08:35:38.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:38 smithi179 ceph-mon[36481]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:38.614 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:38 smithi077 ceph-mon[32007]: pgmap v303: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-05-27T08:35:38.614 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:38 smithi077 ceph-mon[32007]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:38.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:38 smithi092 ceph-mon[36049]: pgmap v303: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 KiB/s wr, 14 op/s 2024-05-27T08:35:38.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:38 smithi092 ceph-mon[36049]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 -- 172.21.15.77:0/2995546297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601041a0 msgr2=0x7fdc601081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 --2- 172.21.15.77:0/2995546297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601041a0 0x7fdc601081c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc480099c0 tx=0x7fdc48009cd0 comp rx=0 tx=0).stop 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 -- 172.21.15.77:0/2995546297 shutdown_connections 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 --2- 172.21.15.77:0/2995546297 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc601088f0 0x7fdc60110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 --2- 172.21.15.77:0/2995546297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc601037f0 0x7fdc60103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 --2- 172.21.15.77:0/2995546297 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601041a0 0x7fdc601081c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 -- 172.21.15.77:0/2995546297 >> 172.21.15.77:0/2995546297 conn(0x7fdc6007a6a0 msgr2=0x7fdc6007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 -- 172.21.15.77:0/2995546297 shutdown_connections 2024-05-27T08:35:39.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.819+0000 7fdc665f2700 1 -- 172.21.15.77:0/2995546297 wait complete. 2024-05-27T08:35:39.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc665f2700 1 Processor -- start 2024-05-27T08:35:39.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc665f2700 1 -- start start 2024-05-27T08:35:39.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc665f2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc601037f0 0x7fdc6019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:39.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc665f2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc601041a0 0x7fdc6019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:39.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc665f2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601088f0 0x7fdc601998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc665f2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601136b0 con 0x7fdc601088f0 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc665f2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60113530 con 0x7fdc601037f0 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc665f2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60113830 con 0x7fdc601041a0 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc64b8f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601088f0 0x7fdc601998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc64b8f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601088f0 0x7fdc601998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50794/0 (socket says 172.21.15.77:50794) 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc64b8f700 1 -- 172.21.15.77:0/775653405 learned_addr learned my addr 172.21.15.77:0/775653405 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.820+0000 7fdc57fff700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc601041a0 0x7fdc6019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc5ffff700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc601037f0 0x7fdc6019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc64b8f700 1 -- 172.21.15.77:0/775653405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc601041a0 msgr2=0x7fdc6019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc64b8f700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc601041a0 0x7fdc6019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc64b8f700 1 -- 172.21.15.77:0/775653405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc601037f0 msgr2=0x7fdc6019f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc64b8f700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc601037f0 0x7fdc6019f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:39.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc64b8f700 1 -- 172.21.15.77:0/775653405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6019a010 con 0x7fdc601088f0 2024-05-27T08:35:39.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc64b8f700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601088f0 0x7fdc601998b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdc58007d70 tx=0x7fdc5800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:39.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc5dffb700 1 -- 172.21.15.77:0/775653405 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58016020 con 0x7fdc601088f0 2024-05-27T08:35:39.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc5dffb700 1 -- 172.21.15.77:0/775653405 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5800ad40 con 0x7fdc601088f0 2024-05-27T08:35:39.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6019a300 con 0x7fdc601088f0 2024-05-27T08:35:39.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc5dffb700 1 -- 172.21.15.77:0/775653405 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580115e0 con 0x7fdc601088f0 2024-05-27T08:35:39.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.821+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601a65c0 con 0x7fdc601088f0 2024-05-27T08:35:39.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.822+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6006a230 con 0x7fdc601088f0 2024-05-27T08:35:39.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.823+0000 7fdc5dffb700 1 -- 172.21.15.77:0/775653405 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdc58019020 con 0x7fdc601088f0 2024-05-27T08:35:39.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.824+0000 7fdc5dffb700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdc4c0b2910 0x7fdc4c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:39.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.824+0000 7fdc5ffff700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdc4c0b2910 0x7fdc4c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:39.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.824+0000 7fdc5dffb700 1 -- 172.21.15.77:0/775653405 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fdc580e25d0 con 0x7fdc601088f0 2024-05-27T08:35:39.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.824+0000 7fdc5ffff700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdc4c0b2910 0x7fdc4c0b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005ea0 tx=0x7fdc50005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:39.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:39.827+0000 7fdc5dffb700 1 -- 172.21.15.77:0/775653405 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc58020020 con 0x7fdc601088f0 2024-05-27T08:35:40.069 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.068+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fdc60063980 con 0x7fdc4c0b2910 2024-05-27T08:35:40.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.073+0000 7fdc5dffb700 1 -- 172.21.15.77:0/775653405 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdc60063980 con 0x7fdc4c0b2910 2024-05-27T08:35:40.074 INFO:teuthology.orchestra.run.smithi077.stdout:/volumes/qa/sv_1/a4abe728-ad7d-4a7c-a93a-78a659276dc3 2024-05-27T08:35:40.076 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdc4c0b2910 msgr2=0x7fdc4c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdc4c0b2910 0x7fdc4c0b4dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005ea0 tx=0x7fdc50005e30 comp rx=0 tx=0).stop 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601088f0 msgr2=0x7fdc601998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601088f0 0x7fdc601998b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdc58007d70 tx=0x7fdc5800a340 comp rx=0 tx=0).stop 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 shutdown_connections 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc601088f0 0x7fdc601998b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc601037f0 0x7fdc6019f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdc4c0b2910 0x7fdc4c0b4dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 --2- 172.21.15.77:0/775653405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc601041a0 0x7fdc6019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 >> 172.21.15.77:0/775653405 conn(0x7fdc6007a6a0 msgr2=0x7fdc601015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 shutdown_connections 2024-05-27T08:35:40.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:40.075+0000 7fdc665f2700 1 -- 172.21.15.77:0/775653405 wait complete. 2024-05-27T08:35:40.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:40 smithi077 ceph-mon[32007]: pgmap v304: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-05-27T08:35:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:40 smithi179 ceph-mon[36481]: pgmap v304: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-05-27T08:35:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:40 smithi092 ceph-mon[36049]: pgmap v304: 97 pgs: 97 active+clean; 610 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-05-27T08:35:40.950 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:35:41.196 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:41.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:41 smithi179 ceph-mon[36481]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:41.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:41 smithi077 ceph-mon[32007]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:41.733 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:41 smithi092 ceph-mon[36049]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:35:42.423 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:42 smithi077 ceph-mon[32007]: pgmap v305: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-27T08:35:42.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:42 smithi179 ceph-mon[36481]: pgmap v305: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-27T08:35:42.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.610+0000 7f9f0a84a700 1 -- 172.21.15.77:0/2438842708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f041084b0 msgr2=0x7f9f04108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:42.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.610+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/2438842708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f041084b0 0x7f9f04108890 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009990 tx=0x7f9ef4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.610+0000 7f9f0a84a700 1 -- 172.21.15.77:0/2438842708 shutdown_connections 2024-05-27T08:35:42.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.610+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/2438842708 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04102e30 0x7f9f0410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.610+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/2438842708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f041084b0 0x7f9f04108890 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.610+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/2438842708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f04102470 0x7f9f041028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.610+0000 7f9f0a84a700 1 -- 172.21.15.77:0/2438842708 >> 172.21.15.77:0/2438842708 conn(0x7f9f04079410 msgr2=0x7f9f04079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:42.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.610+0000 7f9f0a84a700 1 -- 172.21.15.77:0/2438842708 shutdown_connections 2024-05-27T08:35:42.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.611+0000 7f9f0a84a700 1 -- 172.21.15.77:0/2438842708 wait complete. 2024-05-27T08:35:42.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.611+0000 7f9f0a84a700 1 Processor -- start 2024-05-27T08:35:42.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.611+0000 7f9f0a84a700 1 -- start start 2024-05-27T08:35:42.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.611+0000 7f9f0a84a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f04102470 0x7f9f0419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.611+0000 7f9f0a84a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04102e30 0x7f9f0419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.611+0000 7f9f0a84a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f041084b0 0x7f9f04198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.611+0000 7f9f0a84a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9f04112640 con 0x7f9f04102470 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f0a84a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9f041124c0 con 0x7f9f04102e30 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f0a84a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9f041127c0 con 0x7f9f041084b0 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f04102470 0x7f9f0419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f04102470 0x7f9f0419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50806/0 (socket says 172.21.15.77:50806) 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 -- 172.21.15.77:0/1824692808 learned_addr learned my addr 172.21.15.77:0/1824692808 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f08de7700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f041084b0 0x7f9f04198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f037fe700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04102e30 0x7f9f0419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 -- 172.21.15.77:0/1824692808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f041084b0 msgr2=0x7f9f04198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f041084b0 0x7f9f04198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 -- 172.21.15.77:0/1824692808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04102e30 msgr2=0x7f9f0419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04102e30 0x7f9f0419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 -- 172.21.15.77:0/1824692808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f04199080 con 0x7f9f04102470 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f03fff700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f04102470 0x7f9f0419e760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000c00 tx=0x7f9ef40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f017fa700 1 -- 172.21.15.77:0/1824692808 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4027070 con 0x7f9f04102470 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f017fa700 1 -- 172.21.15.77:0/1824692808 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef40041d0 con 0x7f9f04102470 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f04199310 con 0x7f9f04102470 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f017fa700 1 -- 172.21.15.77:0/1824692808 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef400f460 con 0x7f9f04102470 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.612+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0406bc90 con 0x7f9f04102470 2024-05-27T08:35:42.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.613+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f04103b60 con 0x7f9f04102470 2024-05-27T08:35:42.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.614+0000 7f9f017fa700 1 -- 172.21.15.77:0/1824692808 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9ef402a070 con 0x7f9f04102470 2024-05-27T08:35:42.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.615+0000 7f9f017fa700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9ef00b2910 0x7f9ef00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:42.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.615+0000 7f9f037fe700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9ef00b2910 0x7f9ef00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:42.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.615+0000 7f9f017fa700 1 -- 172.21.15.77:0/1824692808 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f9ef40e3630 con 0x7f9f04102470 2024-05-27T08:35:42.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.616+0000 7f9f037fe700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9ef00b2910 0x7f9ef00b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9eec006fd0 tx=0x7f9eec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:42.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.619+0000 7f9f017fa700 1 -- 172.21.15.77:0/1824692808 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9ef4031070 con 0x7f9f04102470 2024-05-27T08:35:42.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:42 smithi092 ceph-mon[36049]: pgmap v305: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-05-27T08:35:42.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.865+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f0406a230 con 0x7f9f04102470 2024-05-27T08:35:42.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.866+0000 7f9f017fa700 1 -- 172.21.15.77:0/1824692808 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11927 (secure 0 0 0) 0x7f9f0406a230 con 0x7f9f04102470 2024-05-27T08:35:42.868 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:35:42.868 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:35:26.135167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24433,24412,24410,24418,24434]},"id":1}]} 2024-05-27T08:35:42.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.868+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9ef00b2910 msgr2=0x7f9ef00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:42.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.868+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9ef00b2910 0x7f9ef00b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9eec006fd0 tx=0x7f9eec008040 comp rx=0 tx=0).stop 2024-05-27T08:35:42.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f04102470 msgr2=0x7f9f0419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:42.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f04102470 0x7f9f0419e760 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000c00 tx=0x7f9ef40047b0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 shutdown_connections 2024-05-27T08:35:42.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f9ef00b2910 0x7f9ef00b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f041084b0 0x7f9f04198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9f04102470 0x7f9f0419e760 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 --2- 172.21.15.77:0/1824692808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9f04102e30 0x7f9f0419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:42.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 >> 172.21.15.77:0/1824692808 conn(0x7f9f04079410 msgr2=0x7f9f04100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:42.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 shutdown_connections 2024-05-27T08:35:42.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:42.869+0000 7f9f0a84a700 1 -- 172.21.15.77:0/1824692808 wait complete. 2024-05-27T08:35:42.871 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-05-27T08:35:43.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:43 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1824692808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:35:43.607 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:43 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1824692808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:35:43.633 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:35:43.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:43 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1824692808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:35:43.878 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:44.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:44 smithi179 ceph-mon[36481]: pgmap v306: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-05-27T08:35:44.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:44 smithi077 ceph-mon[32007]: pgmap v306: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-05-27T08:35:44.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:44 smithi092 ceph-mon[36049]: pgmap v306: 97 pgs: 97 active+clean; 633 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3688409495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 msgr2=0x7fb6c01028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3688409495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 0x7fb6c01028c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8009990 tx=0x7fb6a8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3688409495 shutdown_connections 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3688409495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c0102e00 0x7fb6c010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3688409495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c0108520 0x7fb6c0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3688409495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 0x7fb6c01028c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3688409495 >> 172.21.15.77:0/3688409495 conn(0x7fb6c00793f0 msgr2=0x7fb6c0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3688409495 shutdown_connections 2024-05-27T08:35:45.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.451+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3688409495 wait complete. 2024-05-27T08:35:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6c5cf1700 1 Processor -- start 2024-05-27T08:35:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6c5cf1700 1 -- start start 2024-05-27T08:35:45.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6c5cf1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 0x7fb6c0199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6c5cf1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c0102e00 0x7fb6c019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6c5cf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c0108520 0x7fb6c019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6c5cf1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c01126d0 con 0x7fb6c0102440 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6c5cf1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0112550 con 0x7fb6c0108520 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6c5cf1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0112850 con 0x7fb6c0102e00 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6bf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 0x7fb6c0199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6bf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 0x7fb6c0199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50830/0 (socket says 172.21.15.77:50830) 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.452+0000 7fb6bf7fe700 1 -- 172.21.15.77:0/3842773676 learned_addr learned my addr 172.21.15.77:0/3842773676 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bffff700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c0108520 0x7fb6c019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bf7fe700 1 -- 172.21.15.77:0/3842773676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c0102e00 msgr2=0x7fb6c019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6b7fff700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c0102e00 0x7fb6c019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bf7fe700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c0102e00 0x7fb6c019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bf7fe700 1 -- 172.21.15.77:0/3842773676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c0108520 msgr2=0x7fb6c019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bf7fe700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c0108520 0x7fb6c019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bf7fe700 1 -- 172.21.15.77:0/3842773676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c019edd0 con 0x7fb6c0102440 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bffff700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c0108520 0x7fb6c019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bf7fe700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 0x7fb6c0199e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000b530 tx=0x7fb6b000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bd7fa700 1 -- 172.21.15.77:0/3842773676 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0015020 con 0x7fb6c0102440 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c019f0c0 con 0x7fb6c0102440 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bd7fa700 1 -- 172.21.15.77:0/3842773676 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6b0011ce0 con 0x7fb6c0102440 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6bd7fa700 1 -- 172.21.15.77:0/3842773676 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0010d20 con 0x7fb6c0102440 2024-05-27T08:35:45.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.453+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c01a6940 con 0x7fb6c0102440 2024-05-27T08:35:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.454+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c006a230 con 0x7fb6c0102440 2024-05-27T08:35:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.455+0000 7fb6bd7fa700 1 -- 172.21.15.77:0/3842773676 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb6b0018020 con 0x7fb6c0102440 2024-05-27T08:35:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.456+0000 7fb6bd7fa700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb6ac0b2910 0x7fb6ac0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:45.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.456+0000 7fb6b7fff700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb6ac0b2910 0x7fb6ac0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.456+0000 7fb6bd7fa700 1 -- 172.21.15.77:0/3842773676 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7fb6b00e1950 con 0x7fb6c0102440 2024-05-27T08:35:45.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.456+0000 7fb6b7fff700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb6ac0b2910 0x7fb6ac0b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8005cd0 tx=0x7fb6a8005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:45.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.460+0000 7fb6bd7fa700 1 -- 172.21.15.77:0/3842773676 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb6b001f020 con 0x7fb6c0102440 2024-05-27T08:35:45.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.672+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6c019b120 con 0x7fb6c0102440 2024-05-27T08:35:45.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.673+0000 7fb6bd7fa700 1 -- 172.21.15.77:0/3842773676 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22195 (secure 0 0 0) 0x7fb6b00a97d0 con 0x7fb6c0102440 2024-05-27T08:35:45.675 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:35:45.675 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":83,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:35:08.767247+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.675+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb6ac0b2910 msgr2=0x7fb6ac0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.675+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb6ac0b2910 0x7fb6ac0b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8005cd0 tx=0x7fb6a8005c40 comp rx=0 tx=0).stop 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.675+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 msgr2=0x7fb6c0199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.675+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 0x7fb6c0199e10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000b530 tx=0x7fb6b000b8f0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.675+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 shutdown_connections 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.675+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb6c0108520 0x7fb6c019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.675+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6c0102440 0x7fb6c0199e10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.675+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fb6ac0b2910 0x7fb6ac0b4dd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.676+0000 7fb6c5cf1700 1 --2- 172.21.15.77:0/3842773676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6c0102e00 0x7fb6c019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:45.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.676+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 >> 172.21.15.77:0/3842773676 conn(0x7fb6c00793f0 msgr2=0x7fb6c006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:45.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.676+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 shutdown_connections 2024-05-27T08:35:45.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:45.676+0000 7fb6c5cf1700 1 -- 172.21.15.77:0/3842773676 wait complete. 2024-05-27T08:35:46.461 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T08:35:46.461 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T08:35:46.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:46 smithi179 ceph-mon[36481]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-05-27T08:35:46.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:46 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3842773676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:46.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:46 smithi077 ceph-mon[32007]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-05-27T08:35:46.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:46 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3842773676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:46.702 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:46.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:46 smithi092 ceph-mon[36049]: pgmap v307: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-05-27T08:35:46.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:46 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3842773676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:35:48.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 -- 172.21.15.77:0/1782513837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8104200 msgr2=0x7f6db81081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:48.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/1782513837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8104200 0x7f6db81081f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0099c0 tx=0x7f6dac009cd0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 -- 172.21.15.77:0/1782513837 shutdown_connections 2024-05-27T08:35:48.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/1782513837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db8108920 0x7f6db81104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/1782513837 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db8103850 0x7f6db8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/1782513837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8104200 0x7f6db81081f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 -- 172.21.15.77:0/1782513837 >> 172.21.15.77:0/1782513837 conn(0x7f6db807a620 msgr2=0x7f6db807aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:48.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 -- 172.21.15.77:0/1782513837 shutdown_connections 2024-05-27T08:35:48.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.274+0000 7f6dbca4f700 1 -- 172.21.15.77:0/1782513837 wait complete. 2024-05-27T08:35:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6dbca4f700 1 Processor -- start 2024-05-27T08:35:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6dbca4f700 1 -- start start 2024-05-27T08:35:48.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6dbca4f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db8103850 0x7f6db8075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6dbca4f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db8104200 0x7f6db8075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6dbca4f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8108920 0x7f6db80762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6dbca4f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8113700 con 0x7f6db8108920 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6dbca4f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8113580 con 0x7f6db8103850 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6dbca4f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6db8113880 con 0x7f6db8104200 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6db6d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8108920 0x7f6db80762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6db6d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8108920 0x7f6db80762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50848/0 (socket says 172.21.15.77:50848) 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6db6d9d700 1 -- 172.21.15.77:0/3890878302 learned_addr learned my addr 172.21.15.77:0/3890878302 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.275+0000 7f6db659c700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db8103850 0x7f6db8075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6db5d9b700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db8104200 0x7f6db8075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6db6d9d700 1 -- 172.21.15.77:0/3890878302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db8104200 msgr2=0x7f6db8075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6db6d9d700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db8104200 0x7f6db8075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6db6d9d700 1 -- 172.21.15.77:0/3890878302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db8103850 msgr2=0x7f6db8075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6db6d9d700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db8103850 0x7f6db8075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6db6d9d700 1 -- 172.21.15.77:0/3890878302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db81a8060 con 0x7f6db8108920 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6db6d9d700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8108920 0x7f6db80762c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6db0007ee0 tx=0x7f6db000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6da77fe700 1 -- 172.21.15.77:0/3890878302 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0004bb0 con 0x7f6db8108920 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6da77fe700 1 -- 172.21.15.77:0/3890878302 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db0004d50 con 0x7f6db8108920 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db81a8350 con 0x7f6db8108920 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6da77fe700 1 -- 172.21.15.77:0/3890878302 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db0005920 con 0x7f6db8108920 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.276+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db81a8860 con 0x7f6db8108920 2024-05-27T08:35:48.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.277+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db806a230 con 0x7f6db8108920 2024-05-27T08:35:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.279+0000 7f6da77fe700 1 -- 172.21.15.77:0/3890878302 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6db001a020 con 0x7f6db8108920 2024-05-27T08:35:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.280+0000 7f6da77fe700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f6da00b2910 0x7f6da00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.280+0000 7f6db659c700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f6da00b2910 0x7f6da00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.280+0000 7f6da77fe700 1 -- 172.21.15.77:0/3890878302 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f6db00e1e50 con 0x7f6db8108920 2024-05-27T08:35:48.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.280+0000 7f6db659c700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f6da00b2910 0x7f6da00b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6da8005950 tx=0x7f6da800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:48.284 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.283+0000 7f6da77fe700 1 -- 172.21.15.77:0/3890878302 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6db0010070 con 0x7f6db8108920 2024-05-27T08:35:48.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.484+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6db802e670 con 0x7f6da00b2910 2024-05-27T08:35:48.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.492+0000 7f6da77fe700 1 -- 172.21.15.77:0/3890878302 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258742 (secure 0 0 0) 0x7f6db802e670 con 0x7f6da00b2910 2024-05-27T08:35:48.497 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f6da00b2910 msgr2=0x7f6da00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f6da00b2910 0x7f6da00b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6da8005950 tx=0x7f6da800b410 comp rx=0 tx=0).stop 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8108920 msgr2=0x7f6db80762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8108920 0x7f6db80762c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6db0007ee0 tx=0x7f6db000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 shutdown_connections 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6db8108920 0x7f6db80762c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6db8103850 0x7f6db8075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f6da00b2910 0x7f6da00b4dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 --2- 172.21.15.77:0/3890878302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db8104200 0x7f6db8075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 >> 172.21.15.77:0/3890878302 conn(0x7f6db807a620 msgr2=0x7f6db8102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:48.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 shutdown_connections 2024-05-27T08:35:48.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:48.495+0000 7f6dbca4f700 1 -- 172.21.15.77:0/3890878302 wait complete. 2024-05-27T08:35:48.500 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-05-27T08:35:48.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:48 smithi179 ceph-mon[36481]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-27T08:35:48.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:48 smithi077 ceph-mon[32007]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-27T08:35:48.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:48 smithi092 ceph-mon[36049]: pgmap v308: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-05-27T08:35:49.163 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-05-27T08:35:47.157231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652173,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":328,"num_read_kb":259,"num_write":364,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":300,"ondisk_log_size":300,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361872,"kb_used_data":12720,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579952,"statfs":{"total":1151940427776,"available":1151569870848,"internally_reserved":0,"allocated":13025280,"data_stored":7113663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27539,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":108,"num_read_kb":78,"num_write":20,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003320"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050732+0000","last_change":"2024-05-27T08:35:07.046361+0000","last_active":"2024-05-27T08:35:09.050732+0000","last_peered":"2024-05-27T08:35:09.050732+0000","last_clean":"2024-05-27T08:35:09.050732+0000","last_became_active":"2024-05-27T08:35:07.045954+0000","last_became_peered":"2024-05-27T08:35:07.045954+0000","last_unstale":"2024-05-27T08:35:09.050732+0000","last_undegraded":"2024-05-27T08:35:09.050732+0000","last_fullsized":"2024-05-27T08:35:09.050732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:03:47.458964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054562+0000","last_change":"2024-05-27T08:35:07.043725+0000","last_active":"2024-05-27T08:35:09.054562+0000","last_peered":"2024-05-27T08:35:09.054562+0000","last_clean":"2024-05-27T08:35:09.054562+0000","last_became_active":"2024-05-27T08:35:07.043447+0000","last_became_peered":"2024-05-27T08:35:07.043447+0000","last_unstale":"2024-05-27T08:35:09.054562+0000","last_undegraded":"2024-05-27T08:35:09.054562+0000","last_fullsized":"2024-05-27T08:35:09.054562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:44.509202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054205+0000","last_change":"2024-05-27T08:35:07.048075+0000","last_active":"2024-05-27T08:35:09.054205+0000","last_peered":"2024-05-27T08:35:09.054205+0000","last_clean":"2024-05-27T08:35:09.054205+0000","last_became_active":"2024-05-27T08:35:07.047687+0000","last_became_peered":"2024-05-27T08:35:07.047687+0000","last_unstale":"2024-05-27T08:35:09.054205+0000","last_undegraded":"2024-05-27T08:35:09.054205+0000","last_fullsized":"2024-05-27T08:35:09.054205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:05:15.197231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.043754+0000","last_change":"2024-05-27T08:35:07.046809+0000","last_active":"2024-05-27T08:35:09.043754+0000","last_peered":"2024-05-27T08:35:09.043754+0000","last_clean":"2024-05-27T08:35:09.043754+0000","last_became_active":"2024-05-27T08:35:07.046453+0000","last_became_peered":"2024-05-27T08:35:07.046453+0000","last_unstale":"2024-05-27T08:35:09.043754+0000","last_undegraded":"2024-05-27T08:35:09.043754+0000","last_fullsized":"2024-05-27T08:35:09.043754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:43.209613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051367+0000","last_change":"2024-05-27T08:35:07.040242+0000","last_active":"2024-05-27T08:35:09.051367+0000","last_peered":"2024-05-27T08:35:09.051367+0000","last_clean":"2024-05-27T08:35:09.051367+0000","last_became_active":"2024-05-27T08:35:07.039860+0000","last_became_peered":"2024-05-27T08:35:07.039860+0000","last_unstale":"2024-05-27T08:35:09.051367+0000","last_undegraded":"2024-05-27T08:35:09.051367+0000","last_fullsized":"2024-05-27T08:35:09.051367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:49.212799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052073+0000","last_change":"2024-05-27T08:35:07.048176+0000","last_active":"2024-05-27T08:35:09.052073+0000","last_peered":"2024-05-27T08:35:09.052073+0000","last_clean":"2024-05-27T08:35:09.052073+0000","last_became_active":"2024-05-27T08:35:07.047860+0000","last_became_peered":"2024-05-27T08:35:07.047860+0000","last_unstale":"2024-05-27T08:35:09.052073+0000","last_undegraded":"2024-05-27T08:35:09.052073+0000","last_fullsized":"2024-05-27T08:35:09.052073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:19.459704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049349+0000","last_change":"2024-05-27T08:35:07.052285+0000","last_active":"2024-05-27T08:35:09.049349+0000","last_peered":"2024-05-27T08:35:09.049349+0000","last_clean":"2024-05-27T08:35:09.049349+0000","last_became_active":"2024-05-27T08:35:07.051970+0000","last_became_peered":"2024-05-27T08:35:07.051970+0000","last_unstale":"2024-05-27T08:35:09.049349+0000","last_undegraded":"2024-05-27T08:35:09.049349+0000","last_fullsized":"2024-05-27T08:35:09.049349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:56.543022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050126+0000","last_change":"2024-05-27T08:35:07.047527+0000","last_active":"2024-05-27T08:35:09.050126+0000","last_peered":"2024-05-27T08:35:09.050126+0000","last_clean":"2024-05-27T08:35:09.050126+0000","last_became_active":"2024-05-27T08:35:07.047204+0000","last_became_peered":"2024-05-27T08:35:07.047204+0000","last_unstale":"2024-05-27T08:35:09.050126+0000","last_undegraded":"2024-05-27T08:35:09.050126+0000","last_fullsized":"2024-05-27T08:35:09.050126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:04:43.547847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055946+0000","last_change":"2024-05-27T08:35:07.049108+0000","last_active":"2024-05-27T08:35:09.055946+0000","last_peered":"2024-05-27T08:35:09.055946+0000","last_clean":"2024-05-27T08:35:09.055946+0000","last_became_active":"2024-05-27T08:35:07.048540+0000","last_became_peered":"2024-05-27T08:35:07.048540+0000","last_unstale":"2024-05-27T08:35:09.055946+0000","last_undegraded":"2024-05-27T08:35:09.055946+0000","last_fullsized":"2024-05-27T08:35:09.055946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:56.219362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052611+0000","last_change":"2024-05-27T08:35:07.049775+0000","last_active":"2024-05-27T08:35:09.052611+0000","last_peered":"2024-05-27T08:35:09.052611+0000","last_clean":"2024-05-27T08:35:09.052611+0000","last_became_active":"2024-05-27T08:35:07.049466+0000","last_became_peered":"2024-05-27T08:35:07.049466+0000","last_unstale":"2024-05-27T08:35:09.052611+0000","last_undegraded":"2024-05-27T08:35:09.052611+0000","last_fullsized":"2024-05-27T08:35:09.052611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:18:30.394390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050109+0000","last_change":"2024-05-27T08:35:07.050870+0000","last_active":"2024-05-27T08:35:09.050109+0000","last_peered":"2024-05-27T08:35:09.050109+0000","last_clean":"2024-05-27T08:35:09.050109+0000","last_became_active":"2024-05-27T08:35:07.050527+0000","last_became_peered":"2024-05-27T08:35:07.050527+0000","last_unstale":"2024-05-27T08:35:09.050109+0000","last_undegraded":"2024-05-27T08:35:09.050109+0000","last_fullsized":"2024-05-27T08:35:09.050109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:56:41.477053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.043671+0000","last_change":"2024-05-27T08:35:07.052955+0000","last_active":"2024-05-27T08:35:09.043671+0000","last_peered":"2024-05-27T08:35:09.043671+0000","last_clean":"2024-05-27T08:35:09.043671+0000","last_became_active":"2024-05-27T08:35:07.052500+0000","last_became_peered":"2024-05-27T08:35:07.052500+0000","last_unstale":"2024-05-27T08:35:09.043671+0000","last_undegraded":"2024-05-27T08:35:09.043671+0000","last_fullsized":"2024-05-27T08:35:09.043671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:25.093408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051940+0000","last_change":"2024-05-27T08:35:07.044866+0000","last_active":"2024-05-27T08:35:09.051940+0000","last_peered":"2024-05-27T08:35:09.051940+0000","last_clean":"2024-05-27T08:35:09.051940+0000","last_became_active":"2024-05-27T08:35:07.044554+0000","last_became_peered":"2024-05-27T08:35:07.044554+0000","last_unstale":"2024-05-27T08:35:09.051940+0000","last_undegraded":"2024-05-27T08:35:09.051940+0000","last_fullsized":"2024-05-27T08:35:09.051940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:45.624970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054503+0000","last_change":"2024-05-27T08:35:07.043985+0000","last_active":"2024-05-27T08:35:09.054503+0000","last_peered":"2024-05-27T08:35:09.054503+0000","last_clean":"2024-05-27T08:35:09.054503+0000","last_became_active":"2024-05-27T08:35:07.043694+0000","last_became_peered":"2024-05-27T08:35:07.043694+0000","last_unstale":"2024-05-27T08:35:09.054503+0000","last_undegraded":"2024-05-27T08:35:09.054503+0000","last_fullsized":"2024-05-27T08:35:09.054503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:22:49.796670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054153+0000","last_change":"2024-05-27T08:35:07.045542+0000","last_active":"2024-05-27T08:35:09.054153+0000","last_peered":"2024-05-27T08:35:09.054153+0000","last_clean":"2024-05-27T08:35:09.054153+0000","last_became_active":"2024-05-27T08:35:07.045209+0000","last_became_peered":"2024-05-27T08:35:07.045209+0000","last_unstale":"2024-05-27T08:35:09.054153+0000","last_undegraded":"2024-05-27T08:35:09.054153+0000","last_fullsized":"2024-05-27T08:35:09.054153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:57.733089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054230+0000","last_change":"2024-05-27T08:35:07.047193+0000","last_active":"2024-05-27T08:35:09.054230+0000","last_peered":"2024-05-27T08:35:09.054230+0000","last_clean":"2024-05-27T08:35:09.054230+0000","last_became_active":"2024-05-27T08:35:07.046862+0000","last_became_peered":"2024-05-27T08:35:07.046862+0000","last_unstale":"2024-05-27T08:35:09.054230+0000","last_undegraded":"2024-05-27T08:35:09.054230+0000","last_fullsized":"2024-05-27T08:35:09.054230+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:21.288642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050766+0000","last_change":"2024-05-27T08:35:07.045452+0000","last_active":"2024-05-27T08:35:09.050766+0000","last_peered":"2024-05-27T08:35:09.050766+0000","last_clean":"2024-05-27T08:35:09.050766+0000","last_became_active":"2024-05-27T08:35:07.044783+0000","last_became_peered":"2024-05-27T08:35:07.044783+0000","last_unstale":"2024-05-27T08:35:09.050766+0000","last_undegraded":"2024-05-27T08:35:09.050766+0000","last_fullsized":"2024-05-27T08:35:09.050766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:03.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042696+0000","last_change":"2024-05-27T08:35:07.042864+0000","last_active":"2024-05-27T08:35:09.042696+0000","last_peered":"2024-05-27T08:35:09.042696+0000","last_clean":"2024-05-27T08:35:09.042696+0000","last_became_active":"2024-05-27T08:35:07.042523+0000","last_became_peered":"2024-05-27T08:35:07.042523+0000","last_unstale":"2024-05-27T08:35:09.042696+0000","last_undegraded":"2024-05-27T08:35:09.042696+0000","last_fullsized":"2024-05-27T08:35:09.042696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:34.275977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042875+0000","last_change":"2024-05-27T08:35:07.042667+0000","last_active":"2024-05-27T08:35:09.042875+0000","last_peered":"2024-05-27T08:35:09.042875+0000","last_clean":"2024-05-27T08:35:09.042875+0000","last_became_active":"2024-05-27T08:35:07.042090+0000","last_became_peered":"2024-05-27T08:35:07.042090+0000","last_unstale":"2024-05-27T08:35:09.042875+0000","last_undegraded":"2024-05-27T08:35:09.042875+0000","last_fullsized":"2024-05-27T08:35:09.042875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:28.764641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:41.301328+0000","last_change":"2024-05-27T08:35:04.021045+0000","last_active":"2024-05-27T08:35:41.301328+0000","last_peered":"2024-05-27T08:35:41.301328+0000","last_clean":"2024-05-27T08:35:41.301328+0000","last_became_active":"2024-05-27T08:35:04.020721+0000","last_became_peered":"2024-05-27T08:35:04.020721+0000","last_unstale":"2024-05-27T08:35:41.301328+0000","last_undegraded":"2024-05-27T08:35:41.301328+0000","last_fullsized":"2024-05-27T08:35:41.301328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:42:02.954131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050163+0000","last_change":"2024-05-27T08:35:07.049605+0000","last_active":"2024-05-27T08:35:09.050163+0000","last_peered":"2024-05-27T08:35:09.050163+0000","last_clean":"2024-05-27T08:35:09.050163+0000","last_became_active":"2024-05-27T08:35:07.049307+0000","last_became_peered":"2024-05-27T08:35:07.049307+0000","last_unstale":"2024-05-27T08:35:09.050163+0000","last_undegraded":"2024-05-27T08:35:09.050163+0000","last_fullsized":"2024-05-27T08:35:09.050163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:37:19.286030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054638+0000","last_change":"2024-05-27T08:35:04.245013+0000","last_active":"2024-05-27T08:35:09.054638+0000","last_peered":"2024-05-27T08:35:09.054638+0000","last_clean":"2024-05-27T08:35:09.054638+0000","last_became_active":"2024-05-27T08:35:04.244629+0000","last_became_peered":"2024-05-27T08:35:04.244629+0000","last_unstale":"2024-05-27T08:35:09.054638+0000","last_undegraded":"2024-05-27T08:35:09.054638+0000","last_fullsized":"2024-05-27T08:35:09.054638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:41:43.747592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055065+0000","last_change":"2024-05-27T08:35:07.041776+0000","last_active":"2024-05-27T08:35:09.055065+0000","last_peered":"2024-05-27T08:35:09.055065+0000","last_clean":"2024-05-27T08:35:09.055065+0000","last_became_active":"2024-05-27T08:35:07.041400+0000","last_became_peered":"2024-05-27T08:35:07.041400+0000","last_unstale":"2024-05-27T08:35:09.055065+0000","last_undegraded":"2024-05-27T08:35:09.055065+0000","last_fullsized":"2024-05-27T08:35:09.055065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:10.091310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:24.269937+0000","last_change":"2024-05-27T08:35:04.023072+0000","last_active":"2024-05-27T08:35:24.269937+0000","last_peered":"2024-05-27T08:35:24.269937+0000","last_clean":"2024-05-27T08:35:24.269937+0000","last_became_active":"2024-05-27T08:35:04.022232+0000","last_became_peered":"2024-05-27T08:35:04.022232+0000","last_unstale":"2024-05-27T08:35:24.269937+0000","last_undegraded":"2024-05-27T08:35:24.269937+0000","last_fullsized":"2024-05-27T08:35:24.269937+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:49:57.240621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055373+0000","last_change":"2024-05-27T08:35:07.044727+0000","last_active":"2024-05-27T08:35:09.055373+0000","last_peered":"2024-05-27T08:35:09.055373+0000","last_clean":"2024-05-27T08:35:09.055373+0000","last_became_active":"2024-05-27T08:35:07.044352+0000","last_became_peered":"2024-05-27T08:35:07.044352+0000","last_unstale":"2024-05-27T08:35:09.055373+0000","last_undegraded":"2024-05-27T08:35:09.055373+0000","last_fullsized":"2024-05-27T08:35:09.055373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:52:43.473606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.275112+0000","last_change":"2024-05-27T08:35:04.019179+0000","last_active":"2024-05-27T08:35:23.275112+0000","last_peered":"2024-05-27T08:35:23.275112+0000","last_clean":"2024-05-27T08:35:23.275112+0000","last_became_active":"2024-05-27T08:35:04.018726+0000","last_became_peered":"2024-05-27T08:35:04.018726+0000","last_unstale":"2024-05-27T08:35:23.275112+0000","last_undegraded":"2024-05-27T08:35:23.275112+0000","last_fullsized":"2024-05-27T08:35:23.275112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:37:47.744001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.057487+0000","last_change":"2024-05-27T08:35:07.055562+0000","last_active":"2024-05-27T08:35:09.057487+0000","last_peered":"2024-05-27T08:35:09.057487+0000","last_clean":"2024-05-27T08:35:09.057487+0000","last_became_active":"2024-05-27T08:35:07.055209+0000","last_became_peered":"2024-05-27T08:35:07.055209+0000","last_unstale":"2024-05-27T08:35:09.057487+0000","last_undegraded":"2024-05-27T08:35:09.057487+0000","last_fullsized":"2024-05-27T08:35:09.057487+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:04.920534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:42.304205+0000","last_change":"2024-05-27T08:35:04.022778+0000","last_active":"2024-05-27T08:35:42.304205+0000","last_peered":"2024-05-27T08:35:42.304205+0000","last_clean":"2024-05-27T08:35:42.304205+0000","last_became_active":"2024-05-27T08:35:04.022311+0000","last_became_peered":"2024-05-27T08:35:04.022311+0000","last_unstale":"2024-05-27T08:35:42.304205+0000","last_undegraded":"2024-05-27T08:35:42.304205+0000","last_fullsized":"2024-05-27T08:35:42.304205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:48.174774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050700+0000","last_change":"2024-05-27T08:35:07.040182+0000","last_active":"2024-05-27T08:35:09.050700+0000","last_peered":"2024-05-27T08:35:09.050700+0000","last_clean":"2024-05-27T08:35:09.050700+0000","last_became_active":"2024-05-27T08:35:07.039800+0000","last_became_peered":"2024-05-27T08:35:07.039800+0000","last_unstale":"2024-05-27T08:35:09.050700+0000","last_undegraded":"2024-05-27T08:35:09.050700+0000","last_fullsized":"2024-05-27T08:35:09.050700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:53:53.470284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:22.258772+0000","last_change":"2024-05-27T08:35:04.023757+0000","last_active":"2024-05-27T08:35:22.258772+0000","last_peered":"2024-05-27T08:35:22.258772+0000","last_clean":"2024-05-27T08:35:22.258772+0000","last_became_active":"2024-05-27T08:35:04.023428+0000","last_became_peered":"2024-05-27T08:35:04.023428+0000","last_unstale":"2024-05-27T08:35:22.258772+0000","last_undegraded":"2024-05-27T08:35:22.258772+0000","last_fullsized":"2024-05-27T08:35:22.258772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:45.669006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.056452+0000","last_change":"2024-05-27T08:35:07.051854+0000","last_active":"2024-05-27T08:35:09.056452+0000","last_peered":"2024-05-27T08:35:09.056452+0000","last_clean":"2024-05-27T08:35:09.056452+0000","last_became_active":"2024-05-27T08:35:07.051468+0000","last_became_peered":"2024-05-27T08:35:07.051468+0000","last_unstale":"2024-05-27T08:35:09.056452+0000","last_undegraded":"2024-05-27T08:35:09.056452+0000","last_fullsized":"2024-05-27T08:35:09.056452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:58.998776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:21.253807+0000","last_change":"2024-05-27T08:35:04.015581+0000","last_active":"2024-05-27T08:35:21.253807+0000","last_peered":"2024-05-27T08:35:21.253807+0000","last_clean":"2024-05-27T08:35:21.253807+0000","last_became_active":"2024-05-27T08:35:04.015215+0000","last_became_peered":"2024-05-27T08:35:04.015215+0000","last_unstale":"2024-05-27T08:35:21.253807+0000","last_undegraded":"2024-05-27T08:35:21.253807+0000","last_fullsized":"2024-05-27T08:35:21.253807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:21:51.942008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050014+0000","last_change":"2024-05-27T08:35:07.049798+0000","last_active":"2024-05-27T08:35:09.050014+0000","last_peered":"2024-05-27T08:35:09.050014+0000","last_clean":"2024-05-27T08:35:09.050014+0000","last_became_active":"2024-05-27T08:35:07.048686+0000","last_became_peered":"2024-05-27T08:35:07.048686+0000","last_unstale":"2024-05-27T08:35:09.050014+0000","last_undegraded":"2024-05-27T08:35:09.050014+0000","last_fullsized":"2024-05-27T08:35:09.050014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:39:23.817581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.274337+0000","last_change":"2024-05-27T08:35:04.023826+0000","last_active":"2024-05-27T08:35:23.274337+0000","last_peered":"2024-05-27T08:35:23.274337+0000","last_clean":"2024-05-27T08:35:23.274337+0000","last_became_active":"2024-05-27T08:35:04.023410+0000","last_became_peered":"2024-05-27T08:35:04.023410+0000","last_unstale":"2024-05-27T08:35:23.274337+0000","last_undegraded":"2024-05-27T08:35:23.274337+0000","last_fullsized":"2024-05-27T08:35:23.274337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:38:47.855186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042256+0000","last_change":"2024-05-27T08:35:07.038810+0000","last_active":"2024-05-27T08:35:09.042256+0000","last_peered":"2024-05-27T08:35:09.042256+0000","last_clean":"2024-05-27T08:35:09.042256+0000","last_became_active":"2024-05-27T08:35:07.038480+0000","last_became_peered":"2024-05-27T08:35:07.038480+0000","last_unstale":"2024-05-27T08:35:09.042256+0000","last_undegraded":"2024-05-27T08:35:09.042256+0000","last_fullsized":"2024-05-27T08:35:09.042256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:14.429081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":68,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:44.305784+0000","last_change":"2024-05-27T08:35:04.026934+0000","last_active":"2024-05-27T08:35:44.305784+0000","last_peered":"2024-05-27T08:35:44.305784+0000","last_clean":"2024-05-27T08:35:44.305784+0000","last_became_active":"2024-05-27T08:35:04.026577+0000","last_became_peered":"2024-05-27T08:35:04.026577+0000","last_unstale":"2024-05-27T08:35:44.305784+0000","last_undegraded":"2024-05-27T08:35:44.305784+0000","last_fullsized":"2024-05-27T08:35:44.305784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:59:56.067856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055396+0000","last_change":"2024-05-27T08:35:07.043840+0000","last_active":"2024-05-27T08:35:09.055396+0000","last_peered":"2024-05-27T08:35:09.055396+0000","last_clean":"2024-05-27T08:35:09.055396+0000","last_became_active":"2024-05-27T08:35:07.043224+0000","last_became_peered":"2024-05-27T08:35:07.043224+0000","last_unstale":"2024-05-27T08:35:09.055396+0000","last_undegraded":"2024-05-27T08:35:09.055396+0000","last_fullsized":"2024-05-27T08:35:09.055396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:20.212038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.274717+0000","last_change":"2024-05-27T08:35:04.022602+0000","last_active":"2024-05-27T08:35:23.274717+0000","last_peered":"2024-05-27T08:35:23.274717+0000","last_clean":"2024-05-27T08:35:23.274717+0000","last_became_active":"2024-05-27T08:35:04.022203+0000","last_became_peered":"2024-05-27T08:35:04.022203+0000","last_unstale":"2024-05-27T08:35:23.274717+0000","last_undegraded":"2024-05-27T08:35:23.274717+0000","last_fullsized":"2024-05-27T08:35:23.274717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:51.490696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054951+0000","last_change":"2024-05-27T08:35:07.047257+0000","last_active":"2024-05-27T08:35:09.054951+0000","last_peered":"2024-05-27T08:35:09.054951+0000","last_clean":"2024-05-27T08:35:09.054951+0000","last_became_active":"2024-05-27T08:35:07.046639+0000","last_became_peered":"2024-05-27T08:35:07.046639+0000","last_unstale":"2024-05-27T08:35:09.054951+0000","last_undegraded":"2024-05-27T08:35:09.054951+0000","last_fullsized":"2024-05-27T08:35:09.054951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:04.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:22.259517+0000","last_change":"2024-05-27T08:35:04.245295+0000","last_active":"2024-05-27T08:35:22.259517+0000","last_peered":"2024-05-27T08:35:22.259517+0000","last_clean":"2024-05-27T08:35:22.259517+0000","last_became_active":"2024-05-27T08:35:04.244927+0000","last_became_peered":"2024-05-27T08:35:04.244927+0000","last_unstale":"2024-05-27T08:35:22.259517+0000","last_undegraded":"2024-05-27T08:35:22.259517+0000","last_fullsized":"2024-05-27T08:35:22.259517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:58.893480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054980+0000","last_change":"2024-05-27T08:35:07.048872+0000","last_active":"2024-05-27T08:35:09.054980+0000","last_peered":"2024-05-27T08:35:09.054980+0000","last_clean":"2024-05-27T08:35:09.054980+0000","last_became_active":"2024-05-27T08:35:07.048488+0000","last_became_peered":"2024-05-27T08:35:07.048488+0000","last_unstale":"2024-05-27T08:35:09.054980+0000","last_undegraded":"2024-05-27T08:35:09.054980+0000","last_fullsized":"2024-05-27T08:35:09.054980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:31.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'28","reported_seq":100,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:44.311047+0000","last_change":"2024-05-27T08:35:04.024718+0000","last_active":"2024-05-27T08:35:44.311047+0000","last_peered":"2024-05-27T08:35:44.311047+0000","last_clean":"2024-05-27T08:35:44.311047+0000","last_became_active":"2024-05-27T08:35:04.024155+0000","last_became_peered":"2024-05-27T08:35:04.024155+0000","last_unstale":"2024-05-27T08:35:44.311047+0000","last_undegraded":"2024-05-27T08:35:44.311047+0000","last_fullsized":"2024-05-27T08:35:44.311047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:22.573337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":50,"num_read_kb":77,"num_write":30,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052984+0000","last_change":"2024-05-27T08:35:07.051342+0000","last_active":"2024-05-27T08:35:09.052984+0000","last_peered":"2024-05-27T08:35:09.052984+0000","last_clean":"2024-05-27T08:35:09.052984+0000","last_became_active":"2024-05-27T08:35:07.051030+0000","last_became_peered":"2024-05-27T08:35:07.051030+0000","last_unstale":"2024-05-27T08:35:09.052984+0000","last_undegraded":"2024-05-27T08:35:09.052984+0000","last_fullsized":"2024-05-27T08:35:09.052984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:48.245036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:24.275752+0000","last_change":"2024-05-27T08:35:04.023717+0000","last_active":"2024-05-27T08:35:24.275752+0000","last_peered":"2024-05-27T08:35:24.275752+0000","last_clean":"2024-05-27T08:35:24.275752+0000","last_became_active":"2024-05-27T08:35:04.022017+0000","last_became_peered":"2024-05-27T08:35:04.022017+0000","last_unstale":"2024-05-27T08:35:24.275752+0000","last_undegraded":"2024-05-27T08:35:24.275752+0000","last_fullsized":"2024-05-27T08:35:24.275752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:55.834307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050665+0000","last_change":"2024-05-27T08:35:07.043980+0000","last_active":"2024-05-27T08:35:09.050665+0000","last_peered":"2024-05-27T08:35:09.050665+0000","last_clean":"2024-05-27T08:35:09.050665+0000","last_became_active":"2024-05-27T08:35:07.043672+0000","last_became_peered":"2024-05-27T08:35:07.043672+0000","last_unstale":"2024-05-27T08:35:09.050665+0000","last_undegraded":"2024-05-27T08:35:09.050665+0000","last_fullsized":"2024-05-27T08:35:09.050665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:03:30.243966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273424+0000","last_change":"2024-05-27T08:35:04.022731+0000","last_active":"2024-05-27T08:35:23.273424+0000","last_peered":"2024-05-27T08:35:23.273424+0000","last_clean":"2024-05-27T08:35:23.273424+0000","last_became_active":"2024-05-27T08:35:04.022349+0000","last_became_peered":"2024-05-27T08:35:04.022349+0000","last_unstale":"2024-05-27T08:35:23.273424+0000","last_undegraded":"2024-05-27T08:35:23.273424+0000","last_fullsized":"2024-05-27T08:35:23.273424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:38.651520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049580+0000","last_change":"2024-05-27T08:35:07.052623+0000","last_active":"2024-05-27T08:35:09.049580+0000","last_peered":"2024-05-27T08:35:09.049580+0000","last_clean":"2024-05-27T08:35:09.049580+0000","last_became_active":"2024-05-27T08:35:07.052313+0000","last_became_peered":"2024-05-27T08:35:07.052313+0000","last_unstale":"2024-05-27T08:35:09.049580+0000","last_undegraded":"2024-05-27T08:35:09.049580+0000","last_fullsized":"2024-05-27T08:35:09.049580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:24.612523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:42.306752+0000","last_change":"2024-05-27T08:35:04.023756+0000","last_active":"2024-05-27T08:35:42.306752+0000","last_peered":"2024-05-27T08:35:42.306752+0000","last_clean":"2024-05-27T08:35:42.306752+0000","last_became_active":"2024-05-27T08:35:04.023401+0000","last_became_peered":"2024-05-27T08:35:04.023401+0000","last_unstale":"2024-05-27T08:35:42.306752+0000","last_undegraded":"2024-05-27T08:35:42.306752+0000","last_fullsized":"2024-05-27T08:35:42.306752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:34.887246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049269+0000","last_change":"2024-05-27T08:35:07.050720+0000","last_active":"2024-05-27T08:35:09.049269+0000","last_peered":"2024-05-27T08:35:09.049269+0000","last_clean":"2024-05-27T08:35:09.049269+0000","last_became_active":"2024-05-27T08:35:07.050135+0000","last_became_peered":"2024-05-27T08:35:07.050135+0000","last_unstale":"2024-05-27T08:35:09.049269+0000","last_undegraded":"2024-05-27T08:35:09.049269+0000","last_fullsized":"2024-05-27T08:35:09.049269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:40.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273117+0000","last_change":"2024-05-27T08:35:04.245264+0000","last_active":"2024-05-27T08:35:23.273117+0000","last_peered":"2024-05-27T08:35:23.273117+0000","last_clean":"2024-05-27T08:35:23.273117+0000","last_became_active":"2024-05-27T08:35:04.244776+0000","last_became_peered":"2024-05-27T08:35:04.244776+0000","last_unstale":"2024-05-27T08:35:23.273117+0000","last_undegraded":"2024-05-27T08:35:23.273117+0000","last_fullsized":"2024-05-27T08:35:23.273117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:27:55.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055345+0000","last_change":"2024-05-27T08:35:07.050504+0000","last_active":"2024-05-27T08:35:09.055345+0000","last_peered":"2024-05-27T08:35:09.055345+0000","last_clean":"2024-05-27T08:35:09.055345+0000","last_became_active":"2024-05-27T08:35:07.049368+0000","last_became_peered":"2024-05-27T08:35:07.049368+0000","last_unstale":"2024-05-27T08:35:09.055345+0000","last_undegraded":"2024-05-27T08:35:09.055345+0000","last_fullsized":"2024-05-27T08:35:09.055345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:52.996313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:22.248329+0000","last_change":"2024-05-27T08:35:04.023924+0000","last_active":"2024-05-27T08:35:22.248329+0000","last_peered":"2024-05-27T08:35:22.248329+0000","last_clean":"2024-05-27T08:35:22.248329+0000","last_became_active":"2024-05-27T08:35:04.023548+0000","last_became_peered":"2024-05-27T08:35:04.023548+0000","last_unstale":"2024-05-27T08:35:22.248329+0000","last_undegraded":"2024-05-27T08:35:22.248329+0000","last_fullsized":"2024-05-27T08:35:22.248329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:50.441490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050280+0000","last_change":"2024-05-27T08:35:07.051509+0000","last_active":"2024-05-27T08:35:09.050280+0000","last_peered":"2024-05-27T08:35:09.050280+0000","last_clean":"2024-05-27T08:35:09.050280+0000","last_became_active":"2024-05-27T08:35:07.051184+0000","last_became_peered":"2024-05-27T08:35:07.051184+0000","last_unstale":"2024-05-27T08:35:09.050280+0000","last_undegraded":"2024-05-27T08:35:09.050280+0000","last_fullsized":"2024-05-27T08:35:09.050280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:53.033093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042465+0000","last_change":"2024-05-27T08:35:04.245702+0000","last_active":"2024-05-27T08:35:09.042465+0000","last_peered":"2024-05-27T08:35:09.042465+0000","last_clean":"2024-05-27T08:35:09.042465+0000","last_became_active":"2024-05-27T08:35:04.244624+0000","last_became_peered":"2024-05-27T08:35:04.244624+0000","last_unstale":"2024-05-27T08:35:09.042465+0000","last_undegraded":"2024-05-27T08:35:09.042465+0000","last_fullsized":"2024-05-27T08:35:09.042465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:43:11.055147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.449778+0000","last_change":"2024-05-27T08:35:07.039182+0000","last_active":"2024-05-27T08:35:09.449778+0000","last_peered":"2024-05-27T08:35:09.449778+0000","last_clean":"2024-05-27T08:35:09.449778+0000","last_became_active":"2024-05-27T08:35:07.038793+0000","last_became_peered":"2024-05-27T08:35:07.038793+0000","last_unstale":"2024-05-27T08:35:09.449778+0000","last_undegraded":"2024-05-27T08:35:09.449778+0000","last_fullsized":"2024-05-27T08:35:09.449778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:03.839681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:45.306821+0000","last_change":"2024-05-27T08:35:04.023194+0000","last_active":"2024-05-27T08:35:45.306821+0000","last_peered":"2024-05-27T08:35:45.306821+0000","last_clean":"2024-05-27T08:35:45.306821+0000","last_became_active":"2024-05-27T08:35:04.022823+0000","last_became_peered":"2024-05-27T08:35:04.022823+0000","last_unstale":"2024-05-27T08:35:45.306821+0000","last_undegraded":"2024-05-27T08:35:45.306821+0000","last_fullsized":"2024-05-27T08:35:45.306821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:44:28.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051863+0000","last_change":"2024-05-27T08:35:07.050717+0000","last_active":"2024-05-27T08:35:09.051863+0000","last_peered":"2024-05-27T08:35:09.051863+0000","last_clean":"2024-05-27T08:35:09.051863+0000","last_became_active":"2024-05-27T08:35:07.050413+0000","last_became_peered":"2024-05-27T08:35:07.050413+0000","last_unstale":"2024-05-27T08:35:09.051863+0000","last_undegraded":"2024-05-27T08:35:09.051863+0000","last_fullsized":"2024-05-27T08:35:09.051863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:00:27.693141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.058361+0000","last_change":"2024-05-27T08:35:07.053614+0000","last_active":"2024-05-27T08:35:09.058361+0000","last_peered":"2024-05-27T08:35:09.058361+0000","last_clean":"2024-05-27T08:35:09.058361+0000","last_became_active":"2024-05-27T08:35:07.053253+0000","last_became_peered":"2024-05-27T08:35:07.053253+0000","last_unstale":"2024-05-27T08:35:09.058361+0000","last_undegraded":"2024-05-27T08:35:09.058361+0000","last_fullsized":"2024-05-27T08:35:09.058361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:15:08.985984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:42.306210+0000","last_change":"2024-05-27T08:35:04.246500+0000","last_active":"2024-05-27T08:35:42.306210+0000","last_peered":"2024-05-27T08:35:42.306210+0000","last_clean":"2024-05-27T08:35:42.306210+0000","last_became_active":"2024-05-27T08:35:04.244575+0000","last_became_peered":"2024-05-27T08:35:04.244575+0000","last_unstale":"2024-05-27T08:35:42.306210+0000","last_undegraded":"2024-05-27T08:35:42.306210+0000","last_fullsized":"2024-05-27T08:35:42.306210+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:24:10.818057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050867+0000","last_change":"2024-05-27T08:35:07.048488+0000","last_active":"2024-05-27T08:35:09.050867+0000","last_peered":"2024-05-27T08:35:09.050867+0000","last_clean":"2024-05-27T08:35:09.050867+0000","last_became_active":"2024-05-27T08:35:07.048172+0000","last_became_peered":"2024-05-27T08:35:07.048172+0000","last_unstale":"2024-05-27T08:35:09.050867+0000","last_undegraded":"2024-05-27T08:35:09.050867+0000","last_fullsized":"2024-05-27T08:35:09.050867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:01.632944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:24.277934+0000","last_change":"2024-05-27T08:35:04.022807+0000","last_active":"2024-05-27T08:35:24.277934+0000","last_peered":"2024-05-27T08:35:24.277934+0000","last_clean":"2024-05-27T08:35:24.277934+0000","last_became_active":"2024-05-27T08:35:04.021198+0000","last_became_peered":"2024-05-27T08:35:04.021198+0000","last_unstale":"2024-05-27T08:35:24.277934+0000","last_undegraded":"2024-05-27T08:35:24.277934+0000","last_fullsized":"2024-05-27T08:35:24.277934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:42:10.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049544+0000","last_change":"2024-05-27T08:33:45.683600+0000","last_active":"2024-05-27T08:35:09.049544+0000","last_peered":"2024-05-27T08:35:09.049544+0000","last_clean":"2024-05-27T08:35:09.049544+0000","last_became_active":"2024-05-27T08:33:45.677989+0000","last_became_peered":"2024-05-27T08:33:45.677989+0000","last_unstale":"2024-05-27T08:35:09.049544+0000","last_undegraded":"2024-05-27T08:35:09.049544+0000","last_fullsized":"2024-05-27T08:35:09.049544+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_clean_scrub_stamp":"2024-05-27T08:30:25.507727+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:48:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051232+0000","last_change":"2024-05-27T08:35:07.045879+0000","last_active":"2024-05-27T08:35:09.051232+0000","last_peered":"2024-05-27T08:35:09.051232+0000","last_clean":"2024-05-27T08:35:09.051232+0000","last_became_active":"2024-05-27T08:35:07.045522+0000","last_became_peered":"2024-05-27T08:35:07.045522+0000","last_unstale":"2024-05-27T08:35:09.051232+0000","last_undegraded":"2024-05-27T08:35:09.051232+0000","last_fullsized":"2024-05-27T08:35:09.051232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:30.393769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:21.253160+0000","last_change":"2024-05-27T08:35:04.022419+0000","last_active":"2024-05-27T08:35:21.253160+0000","last_peered":"2024-05-27T08:35:21.253160+0000","last_clean":"2024-05-27T08:35:21.253160+0000","last_became_active":"2024-05-27T08:35:04.022039+0000","last_became_peered":"2024-05-27T08:35:04.022039+0000","last_unstale":"2024-05-27T08:35:21.253160+0000","last_undegraded":"2024-05-27T08:35:21.253160+0000","last_fullsized":"2024-05-27T08:35:21.253160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:31.260596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273484+0000","last_change":"2024-05-27T08:35:04.245214+0000","last_active":"2024-05-27T08:35:23.273484+0000","last_peered":"2024-05-27T08:35:23.273484+0000","last_clean":"2024-05-27T08:35:23.273484+0000","last_became_active":"2024-05-27T08:35:04.244768+0000","last_became_peered":"2024-05-27T08:35:04.244768+0000","last_unstale":"2024-05-27T08:35:23.273484+0000","last_undegraded":"2024-05-27T08:35:23.273484+0000","last_fullsized":"2024-05-27T08:35:23.273484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:34.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052346+0000","last_change":"2024-05-27T08:35:07.042721+0000","last_active":"2024-05-27T08:35:09.052346+0000","last_peered":"2024-05-27T08:35:09.052346+0000","last_clean":"2024-05-27T08:35:09.052346+0000","last_became_active":"2024-05-27T08:35:07.042435+0000","last_became_peered":"2024-05-27T08:35:07.042435+0000","last_unstale":"2024-05-27T08:35:09.052346+0000","last_undegraded":"2024-05-27T08:35:09.052346+0000","last_fullsized":"2024-05-27T08:35:09.052346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:49.186753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:39.051204+0000","last_change":"2024-05-27T08:35:04.017995+0000","last_active":"2024-05-27T08:35:39.051204+0000","last_peered":"2024-05-27T08:35:39.051204+0000","last_clean":"2024-05-27T08:35:39.051204+0000","last_became_active":"2024-05-27T08:35:04.017675+0000","last_became_peered":"2024-05-27T08:35:04.017675+0000","last_unstale":"2024-05-27T08:35:39.051204+0000","last_undegraded":"2024-05-27T08:35:39.051204+0000","last_fullsized":"2024-05-27T08:35:39.051204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:23:15.452624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051045+0000","last_change":"2024-05-27T08:35:07.046553+0000","last_active":"2024-05-27T08:35:09.051045+0000","last_peered":"2024-05-27T08:35:09.051045+0000","last_clean":"2024-05-27T08:35:09.051045+0000","last_became_active":"2024-05-27T08:35:07.046229+0000","last_became_peered":"2024-05-27T08:35:07.046229+0000","last_unstale":"2024-05-27T08:35:09.051045+0000","last_undegraded":"2024-05-27T08:35:09.051045+0000","last_fullsized":"2024-05-27T08:35:09.051045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:21.024307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273336+0000","last_change":"2024-05-27T08:35:04.262439+0000","last_active":"2024-05-27T08:35:23.273336+0000","last_peered":"2024-05-27T08:35:23.273336+0000","last_clean":"2024-05-27T08:35:23.273336+0000","last_became_active":"2024-05-27T08:35:04.261958+0000","last_became_peered":"2024-05-27T08:35:04.261958+0000","last_unstale":"2024-05-27T08:35:23.273336+0000","last_undegraded":"2024-05-27T08:35:23.273336+0000","last_fullsized":"2024-05-27T08:35:23.273336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:11:05.763068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054546+0000","last_change":"2024-05-27T08:35:07.042307+0000","last_active":"2024-05-27T08:35:09.054546+0000","last_peered":"2024-05-27T08:35:09.054546+0000","last_clean":"2024-05-27T08:35:09.054546+0000","last_became_active":"2024-05-27T08:35:07.042001+0000","last_became_peered":"2024-05-27T08:35:07.042001+0000","last_unstale":"2024-05-27T08:35:09.054546+0000","last_undegraded":"2024-05-27T08:35:09.054546+0000","last_fullsized":"2024-05-27T08:35:09.054546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:51.030157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:20.240835+0000","last_change":"2024-05-27T08:35:04.245436+0000","last_active":"2024-05-27T08:35:20.240835+0000","last_peered":"2024-05-27T08:35:20.240835+0000","last_clean":"2024-05-27T08:35:20.240835+0000","last_became_active":"2024-05-27T08:35:04.245039+0000","last_became_peered":"2024-05-27T08:35:04.245039+0000","last_unstale":"2024-05-27T08:35:20.240835+0000","last_undegraded":"2024-05-27T08:35:20.240835+0000","last_fullsized":"2024-05-27T08:35:20.240835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:10:11.756864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050358+0000","last_change":"2024-05-27T08:35:07.051524+0000","last_active":"2024-05-27T08:35:09.050358+0000","last_peered":"2024-05-27T08:35:09.050358+0000","last_clean":"2024-05-27T08:35:09.050358+0000","last_became_active":"2024-05-27T08:35:07.050676+0000","last_became_peered":"2024-05-27T08:35:07.050676+0000","last_unstale":"2024-05-27T08:35:09.050358+0000","last_undegraded":"2024-05-27T08:35:09.050358+0000","last_fullsized":"2024-05-27T08:35:09.050358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:06.818977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273411+0000","last_change":"2024-05-27T08:35:04.022064+0000","last_active":"2024-05-27T08:35:23.273411+0000","last_peered":"2024-05-27T08:35:23.273411+0000","last_clean":"2024-05-27T08:35:23.273411+0000","last_became_active":"2024-05-27T08:35:04.021691+0000","last_became_peered":"2024-05-27T08:35:04.021691+0000","last_unstale":"2024-05-27T08:35:23.273411+0000","last_undegraded":"2024-05-27T08:35:23.273411+0000","last_fullsized":"2024-05-27T08:35:23.273411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:54.557756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050151+0000","last_change":"2024-05-27T08:35:07.049666+0000","last_active":"2024-05-27T08:35:09.050151+0000","last_peered":"2024-05-27T08:35:09.050151+0000","last_clean":"2024-05-27T08:35:09.050151+0000","last_became_active":"2024-05-27T08:35:07.049312+0000","last_became_peered":"2024-05-27T08:35:07.049312+0000","last_unstale":"2024-05-27T08:35:09.050151+0000","last_undegraded":"2024-05-27T08:35:09.050151+0000","last_fullsized":"2024-05-27T08:35:09.050151+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:37:53.704582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:22.259907+0000","last_change":"2024-05-27T08:35:04.018457+0000","last_active":"2024-05-27T08:35:22.259907+0000","last_peered":"2024-05-27T08:35:22.259907+0000","last_clean":"2024-05-27T08:35:22.259907+0000","last_became_active":"2024-05-27T08:35:04.018008+0000","last_became_peered":"2024-05-27T08:35:04.018008+0000","last_unstale":"2024-05-27T08:35:22.259907+0000","last_undegraded":"2024-05-27T08:35:22.259907+0000","last_fullsized":"2024-05-27T08:35:22.259907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:08:13.228725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050512+0000","last_change":"2024-05-27T08:35:07.047040+0000","last_active":"2024-05-27T08:35:09.050512+0000","last_peered":"2024-05-27T08:35:09.050512+0000","last_clean":"2024-05-27T08:35:09.050512+0000","last_became_active":"2024-05-27T08:35:07.046731+0000","last_became_peered":"2024-05-27T08:35:07.046731+0000","last_unstale":"2024-05-27T08:35:09.050512+0000","last_undegraded":"2024-05-27T08:35:09.050512+0000","last_fullsized":"2024-05-27T08:35:09.050512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:25.368349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:20.241434+0000","last_change":"2024-05-27T08:35:04.022804+0000","last_active":"2024-05-27T08:35:20.241434+0000","last_peered":"2024-05-27T08:35:20.241434+0000","last_clean":"2024-05-27T08:35:20.241434+0000","last_became_active":"2024-05-27T08:35:04.022458+0000","last_became_peered":"2024-05-27T08:35:04.022458+0000","last_unstale":"2024-05-27T08:35:20.241434+0000","last_undegraded":"2024-05-27T08:35:20.241434+0000","last_fullsized":"2024-05-27T08:35:20.241434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:01.029226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049736+0000","last_change":"2024-05-27T08:35:07.049136+0000","last_active":"2024-05-27T08:35:09.049736+0000","last_peered":"2024-05-27T08:35:09.049736+0000","last_clean":"2024-05-27T08:35:09.049736+0000","last_became_active":"2024-05-27T08:35:07.048793+0000","last_became_peered":"2024-05-27T08:35:07.048793+0000","last_unstale":"2024-05-27T08:35:09.049736+0000","last_undegraded":"2024-05-27T08:35:09.049736+0000","last_fullsized":"2024-05-27T08:35:09.049736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:10:21.191743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":49,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:43.305256+0000","last_change":"2024-05-27T08:35:04.262102+0000","last_active":"2024-05-27T08:35:43.305256+0000","last_peered":"2024-05-27T08:35:43.305256+0000","last_clean":"2024-05-27T08:35:43.305256+0000","last_became_active":"2024-05-27T08:35:04.261665+0000","last_became_peered":"2024-05-27T08:35:04.261665+0000","last_unstale":"2024-05-27T08:35:43.305256+0000","last_undegraded":"2024-05-27T08:35:43.305256+0000","last_fullsized":"2024-05-27T08:35:43.305256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:57:57.712197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050329+0000","last_change":"2024-05-27T08:35:07.051557+0000","last_active":"2024-05-27T08:35:09.050329+0000","last_peered":"2024-05-27T08:35:09.050329+0000","last_clean":"2024-05-27T08:35:09.050329+0000","last_became_active":"2024-05-27T08:35:07.051246+0000","last_became_peered":"2024-05-27T08:35:07.051246+0000","last_unstale":"2024-05-27T08:35:09.050329+0000","last_undegraded":"2024-05-27T08:35:09.050329+0000","last_fullsized":"2024-05-27T08:35:09.050329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:08:13.248301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273758+0000","last_change":"2024-05-27T08:35:04.022328+0000","last_active":"2024-05-27T08:35:23.273758+0000","last_peered":"2024-05-27T08:35:23.273758+0000","last_clean":"2024-05-27T08:35:23.273758+0000","last_became_active":"2024-05-27T08:35:04.021711+0000","last_became_peered":"2024-05-27T08:35:04.021711+0000","last_unstale":"2024-05-27T08:35:23.273758+0000","last_undegraded":"2024-05-27T08:35:23.273758+0000","last_fullsized":"2024-05-27T08:35:23.273758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:32.841474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049729+0000","last_change":"2024-05-27T08:35:07.056768+0000","last_active":"2024-05-27T08:35:09.049729+0000","last_peered":"2024-05-27T08:35:09.049729+0000","last_clean":"2024-05-27T08:35:09.049729+0000","last_became_active":"2024-05-27T08:35:07.056482+0000","last_became_peered":"2024-05-27T08:35:07.056482+0000","last_unstale":"2024-05-27T08:35:09.049729+0000","last_undegraded":"2024-05-27T08:35:09.049729+0000","last_fullsized":"2024-05-27T08:35:09.049729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:23.515625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.271596+0000","last_change":"2024-05-27T08:35:04.016613+0000","last_active":"2024-05-27T08:35:23.271596+0000","last_peered":"2024-05-27T08:35:23.271596+0000","last_clean":"2024-05-27T08:35:23.271596+0000","last_became_active":"2024-05-27T08:35:04.016268+0000","last_became_peered":"2024-05-27T08:35:04.016268+0000","last_unstale":"2024-05-27T08:35:23.271596+0000","last_undegraded":"2024-05-27T08:35:23.271596+0000","last_fullsized":"2024-05-27T08:35:23.271596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:11.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050355+0000","last_change":"2024-05-27T08:35:07.051515+0000","last_active":"2024-05-27T08:35:09.050355+0000","last_peered":"2024-05-27T08:35:09.050355+0000","last_clean":"2024-05-27T08:35:09.050355+0000","last_became_active":"2024-05-27T08:35:07.051197+0000","last_became_peered":"2024-05-27T08:35:07.051197+0000","last_unstale":"2024-05-27T08:35:09.050355+0000","last_undegraded":"2024-05-27T08:35:09.050355+0000","last_fullsized":"2024-05-27T08:35:09.050355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:49.233474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050330+0000","last_change":"2024-05-27T08:35:07.051960+0000","last_active":"2024-05-27T08:35:09.050330+0000","last_peered":"2024-05-27T08:35:09.050330+0000","last_clean":"2024-05-27T08:35:09.050330+0000","last_became_active":"2024-05-27T08:35:07.051395+0000","last_became_peered":"2024-05-27T08:35:07.051395+0000","last_unstale":"2024-05-27T08:35:09.050330+0000","last_undegraded":"2024-05-27T08:35:09.050330+0000","last_fullsized":"2024-05-27T08:35:09.050330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:57.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042574+0000","last_change":"2024-05-27T08:35:07.050096+0000","last_active":"2024-05-27T08:35:09.042574+0000","last_peered":"2024-05-27T08:35:09.042574+0000","last_clean":"2024-05-27T08:35:09.042574+0000","last_became_active":"2024-05-27T08:35:07.049755+0000","last_became_peered":"2024-05-27T08:35:07.049755+0000","last_unstale":"2024-05-27T08:35:09.042574+0000","last_undegraded":"2024-05-27T08:35:09.042574+0000","last_fullsized":"2024-05-27T08:35:09.042574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:39.768128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042494+0000","last_change":"2024-05-27T08:35:07.048281+0000","last_active":"2024-05-27T08:35:09.042494+0000","last_peered":"2024-05-27T08:35:09.042494+0000","last_clean":"2024-05-27T08:35:09.042494+0000","last_became_active":"2024-05-27T08:35:07.047879+0000","last_became_peered":"2024-05-27T08:35:07.047879+0000","last_unstale":"2024-05-27T08:35:09.042494+0000","last_undegraded":"2024-05-27T08:35:09.042494+0000","last_fullsized":"2024-05-27T08:35:09.042494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:43.631671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054655+0000","last_change":"2024-05-27T08:35:07.050501+0000","last_active":"2024-05-27T08:35:09.054655+0000","last_peered":"2024-05-27T08:35:09.054655+0000","last_clean":"2024-05-27T08:35:09.054655+0000","last_became_active":"2024-05-27T08:35:07.050159+0000","last_became_peered":"2024-05-27T08:35:07.050159+0000","last_unstale":"2024-05-27T08:35:09.054655+0000","last_undegraded":"2024-05-27T08:35:09.054655+0000","last_fullsized":"2024-05-27T08:35:09.054655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:17:23.293019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052179+0000","last_change":"2024-05-27T08:35:07.047450+0000","last_active":"2024-05-27T08:35:09.052179+0000","last_peered":"2024-05-27T08:35:09.052179+0000","last_clean":"2024-05-27T08:35:09.052179+0000","last_became_active":"2024-05-27T08:35:07.047094+0000","last_became_peered":"2024-05-27T08:35:07.047094+0000","last_unstale":"2024-05-27T08:35:09.052179+0000","last_undegraded":"2024-05-27T08:35:09.052179+0000","last_fullsized":"2024-05-27T08:35:09.052179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:11.882117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050942+0000","last_change":"2024-05-27T08:35:07.052382+0000","last_active":"2024-05-27T08:35:09.050942+0000","last_peered":"2024-05-27T08:35:09.050942+0000","last_clean":"2024-05-27T08:35:09.050942+0000","last_became_active":"2024-05-27T08:35:07.052049+0000","last_became_peered":"2024-05-27T08:35:07.052049+0000","last_unstale":"2024-05-27T08:35:09.050942+0000","last_undegraded":"2024-05-27T08:35:09.050942+0000","last_fullsized":"2024-05-27T08:35:09.050942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:08.912872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042910+0000","last_change":"2024-05-27T08:35:07.043810+0000","last_active":"2024-05-27T08:35:09.042910+0000","last_peered":"2024-05-27T08:35:09.042910+0000","last_clean":"2024-05-27T08:35:09.042910+0000","last_became_active":"2024-05-27T08:35:07.043449+0000","last_became_peered":"2024-05-27T08:35:07.043449+0000","last_unstale":"2024-05-27T08:35:09.042910+0000","last_undegraded":"2024-05-27T08:35:09.042910+0000","last_fullsized":"2024-05-27T08:35:09.042910+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:18.089405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050822+0000","last_change":"2024-05-27T08:35:07.043676+0000","last_active":"2024-05-27T08:35:09.050822+0000","last_peered":"2024-05-27T08:35:09.050822+0000","last_clean":"2024-05-27T08:35:09.050822+0000","last_became_active":"2024-05-27T08:35:07.043365+0000","last_became_peered":"2024-05-27T08:35:07.043365+0000","last_unstale":"2024-05-27T08:35:09.050822+0000","last_undegraded":"2024-05-27T08:35:09.050822+0000","last_fullsized":"2024-05-27T08:35:09.050822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:32.374612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.058160+0000","last_change":"2024-05-27T08:35:07.049960+0000","last_active":"2024-05-27T08:35:09.058160+0000","last_peered":"2024-05-27T08:35:09.058160+0000","last_clean":"2024-05-27T08:35:09.058160+0000","last_became_active":"2024-05-27T08:35:07.049595+0000","last_became_peered":"2024-05-27T08:35:07.049595+0000","last_unstale":"2024-05-27T08:35:09.058160+0000","last_undegraded":"2024-05-27T08:35:09.058160+0000","last_fullsized":"2024-05-27T08:35:09.058160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:18.182658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050739+0000","last_change":"2024-05-27T08:35:07.043790+0000","last_active":"2024-05-27T08:35:09.050739+0000","last_peered":"2024-05-27T08:35:09.050739+0000","last_clean":"2024-05-27T08:35:09.050739+0000","last_became_active":"2024-05-27T08:35:07.043476+0000","last_became_peered":"2024-05-27T08:35:07.043476+0000","last_unstale":"2024-05-27T08:35:09.050739+0000","last_undegraded":"2024-05-27T08:35:09.050739+0000","last_fullsized":"2024-05-27T08:35:09.050739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:01.626979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054997+0000","last_change":"2024-05-27T08:35:07.044928+0000","last_active":"2024-05-27T08:35:09.054997+0000","last_peered":"2024-05-27T08:35:09.054997+0000","last_clean":"2024-05-27T08:35:09.054997+0000","last_became_active":"2024-05-27T08:35:07.044577+0000","last_became_peered":"2024-05-27T08:35:07.044577+0000","last_unstale":"2024-05-27T08:35:09.054997+0000","last_undegraded":"2024-05-27T08:35:09.054997+0000","last_fullsized":"2024-05-27T08:35:09.054997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:36.560525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052101+0000","last_change":"2024-05-27T08:35:07.045067+0000","last_active":"2024-05-27T08:35:09.052101+0000","last_peered":"2024-05-27T08:35:09.052101+0000","last_clean":"2024-05-27T08:35:09.052101+0000","last_became_active":"2024-05-27T08:35:07.044759+0000","last_became_peered":"2024-05-27T08:35:07.044759+0000","last_unstale":"2024-05-27T08:35:09.052101+0000","last_undegraded":"2024-05-27T08:35:09.052101+0000","last_fullsized":"2024-05-27T08:35:09.052101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:08.440135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050351+0000","last_change":"2024-05-27T08:35:07.048186+0000","last_active":"2024-05-27T08:35:09.050351+0000","last_peered":"2024-05-27T08:35:09.050351+0000","last_clean":"2024-05-27T08:35:09.050351+0000","last_became_active":"2024-05-27T08:35:07.047828+0000","last_became_peered":"2024-05-27T08:35:07.047828+0000","last_unstale":"2024-05-27T08:35:09.050351+0000","last_undegraded":"2024-05-27T08:35:09.050351+0000","last_fullsized":"2024-05-27T08:35:09.050351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:09.928234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61571,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":247,"num_read_kb":190,"num_write":239,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":184713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":430954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743452,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1597440,"data_stored":1076741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939680,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":430884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":431262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1024171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495663,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":430601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":485881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":431940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":486492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117054,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":430426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920824,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":430853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1023458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":56494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T08:35:49.168 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T08:35:49.415 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:49.537 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:49 smithi077 ceph-mon[32007]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:35:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:49 smithi179 ceph-mon[36481]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:35:49.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:49 smithi092 ceph-mon[36049]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:35:50.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:50 smithi179 ceph-mon[36481]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-27T08:35:50.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:50 smithi077 ceph-mon[32007]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-27T08:35:50.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:50 smithi092 ceph-mon[36049]: pgmap v309: 97 pgs: 97 active+clean; 637 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-05-27T08:35:50.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 -- 172.21.15.77:0/1718121312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c102ff0 msgr2=0x7f229c103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:50.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 --2- 172.21.15.77:0/1718121312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c102ff0 0x7f229c103470 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2290009990 tx=0x7f2290009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 -- 172.21.15.77:0/1718121312 shutdown_connections 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 --2- 172.21.15.77:0/1718121312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f229c1039b0 0x7f229c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 --2- 172.21.15.77:0/1718121312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f229c107470 0x7f229c107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 --2- 172.21.15.77:0/1718121312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c102ff0 0x7f229c103470 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 -- 172.21.15.77:0/1718121312 >> 172.21.15.77:0/1718121312 conn(0x7f229c0777e0 msgr2=0x7f229c077bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 -- 172.21.15.77:0/1718121312 shutdown_connections 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.997+0000 7f22a130a700 1 -- 172.21.15.77:0/1718121312 wait complete. 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.998+0000 7f22a130a700 1 Processor -- start 2024-05-27T08:35:51.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.998+0000 7f22a130a700 1 -- start start 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f22a130a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f229c102ff0 0x7f229c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f22a130a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f229c1039b0 0x7f229c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f22a130a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c107470 0x7f229c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f22a130a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f229c112720 con 0x7f229c107470 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f22a130a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1125a0 con 0x7f229c102ff0 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f22a130a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1128a0 con 0x7f229c1039b0 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c107470 0x7f229c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c107470 0x7f229c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49946/0 (socket says 172.21.15.77:49946) 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 -- 172.21.15.77:0/27278553 learned_addr learned my addr 172.21.15.77:0/27278553 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:51.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229a7fc700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f229c1039b0 0x7f229c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229affd700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f229c102ff0 0x7f229c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 -- 172.21.15.77:0/27278553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f229c1039b0 msgr2=0x7f229c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f229c1039b0 0x7f229c19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 -- 172.21.15.77:0/27278553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f229c102ff0 msgr2=0x7f229c19e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f229c102ff0 0x7f229c19e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 -- 172.21.15.77:0/27278553 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c199100 con 0x7f229c107470 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f229b7fe700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c107470 0x7f229c1989a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f229400b9e0 tx=0x7f229400bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f2283fff700 1 -- 172.21.15.77:0/27278553 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229400d8e0 con 0x7f229c107470 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f2283fff700 1 -- 172.21.15.77:0/27278553 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2294015420 con 0x7f229c107470 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f2283fff700 1 -- 172.21.15.77:0/27278553 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294014630 con 0x7f229c107470 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:50.999+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c1993f0 con 0x7f229c107470 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.000+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c1a5660 con 0x7f229c107470 2024-05-27T08:35:51.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.000+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c06a230 con 0x7f229c107470 2024-05-27T08:35:51.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.002+0000 7f2283fff700 1 -- 172.21.15.77:0/27278553 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f22940155d0 con 0x7f229c107470 2024-05-27T08:35:51.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.002+0000 7f2283fff700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f22840b2910 0x7f22840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:51.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.002+0000 7f229affd700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f22840b2910 0x7f22840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:51.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.002+0000 7f2283fff700 1 -- 172.21.15.77:0/27278553 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f22940e1f90 con 0x7f229c107470 2024-05-27T08:35:51.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.003+0000 7f229affd700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f22840b2910 0x7f22840b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f229c1089a0 tx=0x7f228c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:51.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.006+0000 7f2283fff700 1 -- 172.21.15.77:0/27278553 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22940a9d00 con 0x7f229c107470 2024-05-27T08:35:51.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.203+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f229c1005a0 con 0x7f22840b2910 2024-05-27T08:35:51.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.210+0000 7f2283fff700 1 -- 172.21.15.77:0/27278553 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258743 (secure 0 0 0) 0x7f229c1005a0 con 0x7f22840b2910 2024-05-27T08:35:51.215 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f22840b2910 msgr2=0x7f22840b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f22840b2910 0x7f22840b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f229c1089a0 tx=0x7f228c009190 comp rx=0 tx=0).stop 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c107470 msgr2=0x7f229c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c107470 0x7f229c1989a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f229400b9e0 tx=0x7f229400bcf0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 shutdown_connections 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f229c107470 0x7f229c1989a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f229c102ff0 0x7f229c19e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f22840b2910 0x7f22840b4dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 --2- 172.21.15.77:0/27278553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f229c1039b0 0x7f229c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:51.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 >> 172.21.15.77:0/27278553 conn(0x7f229c0777e0 msgr2=0x7f229c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:51.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 shutdown_connections 2024-05-27T08:35:51.218 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:51.214+0000 7f22a130a700 1 -- 172.21.15.77:0/27278553 wait complete. 2024-05-27T08:35:51.218 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-05-27T08:35:51.895 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-05-27T08:35:51.158164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":373,"num_read_kb":289,"num_write":371,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361880,"kb_used_data":12728,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579944,"statfs":{"total":1151940427776,"available":1151569862656,"internally_reserved":0,"allocated":13033472,"data_stored":7124453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32934,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":128,"num_read_kb":98,"num_write":23,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003049"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050732+0000","last_change":"2024-05-27T08:35:07.046361+0000","last_active":"2024-05-27T08:35:09.050732+0000","last_peered":"2024-05-27T08:35:09.050732+0000","last_clean":"2024-05-27T08:35:09.050732+0000","last_became_active":"2024-05-27T08:35:07.045954+0000","last_became_peered":"2024-05-27T08:35:07.045954+0000","last_unstale":"2024-05-27T08:35:09.050732+0000","last_undegraded":"2024-05-27T08:35:09.050732+0000","last_fullsized":"2024-05-27T08:35:09.050732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:03:47.458964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054562+0000","last_change":"2024-05-27T08:35:07.043725+0000","last_active":"2024-05-27T08:35:09.054562+0000","last_peered":"2024-05-27T08:35:09.054562+0000","last_clean":"2024-05-27T08:35:09.054562+0000","last_became_active":"2024-05-27T08:35:07.043447+0000","last_became_peered":"2024-05-27T08:35:07.043447+0000","last_unstale":"2024-05-27T08:35:09.054562+0000","last_undegraded":"2024-05-27T08:35:09.054562+0000","last_fullsized":"2024-05-27T08:35:09.054562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:44.509202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054205+0000","last_change":"2024-05-27T08:35:07.048075+0000","last_active":"2024-05-27T08:35:09.054205+0000","last_peered":"2024-05-27T08:35:09.054205+0000","last_clean":"2024-05-27T08:35:09.054205+0000","last_became_active":"2024-05-27T08:35:07.047687+0000","last_became_peered":"2024-05-27T08:35:07.047687+0000","last_unstale":"2024-05-27T08:35:09.054205+0000","last_undegraded":"2024-05-27T08:35:09.054205+0000","last_fullsized":"2024-05-27T08:35:09.054205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:05:15.197231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.043754+0000","last_change":"2024-05-27T08:35:07.046809+0000","last_active":"2024-05-27T08:35:09.043754+0000","last_peered":"2024-05-27T08:35:09.043754+0000","last_clean":"2024-05-27T08:35:09.043754+0000","last_became_active":"2024-05-27T08:35:07.046453+0000","last_became_peered":"2024-05-27T08:35:07.046453+0000","last_unstale":"2024-05-27T08:35:09.043754+0000","last_undegraded":"2024-05-27T08:35:09.043754+0000","last_fullsized":"2024-05-27T08:35:09.043754+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:43.209613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051367+0000","last_change":"2024-05-27T08:35:07.040242+0000","last_active":"2024-05-27T08:35:09.051367+0000","last_peered":"2024-05-27T08:35:09.051367+0000","last_clean":"2024-05-27T08:35:09.051367+0000","last_became_active":"2024-05-27T08:35:07.039860+0000","last_became_peered":"2024-05-27T08:35:07.039860+0000","last_unstale":"2024-05-27T08:35:09.051367+0000","last_undegraded":"2024-05-27T08:35:09.051367+0000","last_fullsized":"2024-05-27T08:35:09.051367+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:49.212799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052073+0000","last_change":"2024-05-27T08:35:07.048176+0000","last_active":"2024-05-27T08:35:09.052073+0000","last_peered":"2024-05-27T08:35:09.052073+0000","last_clean":"2024-05-27T08:35:09.052073+0000","last_became_active":"2024-05-27T08:35:07.047860+0000","last_became_peered":"2024-05-27T08:35:07.047860+0000","last_unstale":"2024-05-27T08:35:09.052073+0000","last_undegraded":"2024-05-27T08:35:09.052073+0000","last_fullsized":"2024-05-27T08:35:09.052073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:19.459704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049349+0000","last_change":"2024-05-27T08:35:07.052285+0000","last_active":"2024-05-27T08:35:09.049349+0000","last_peered":"2024-05-27T08:35:09.049349+0000","last_clean":"2024-05-27T08:35:09.049349+0000","last_became_active":"2024-05-27T08:35:07.051970+0000","last_became_peered":"2024-05-27T08:35:07.051970+0000","last_unstale":"2024-05-27T08:35:09.049349+0000","last_undegraded":"2024-05-27T08:35:09.049349+0000","last_fullsized":"2024-05-27T08:35:09.049349+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:56.543022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050126+0000","last_change":"2024-05-27T08:35:07.047527+0000","last_active":"2024-05-27T08:35:09.050126+0000","last_peered":"2024-05-27T08:35:09.050126+0000","last_clean":"2024-05-27T08:35:09.050126+0000","last_became_active":"2024-05-27T08:35:07.047204+0000","last_became_peered":"2024-05-27T08:35:07.047204+0000","last_unstale":"2024-05-27T08:35:09.050126+0000","last_undegraded":"2024-05-27T08:35:09.050126+0000","last_fullsized":"2024-05-27T08:35:09.050126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:04:43.547847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055946+0000","last_change":"2024-05-27T08:35:07.049108+0000","last_active":"2024-05-27T08:35:09.055946+0000","last_peered":"2024-05-27T08:35:09.055946+0000","last_clean":"2024-05-27T08:35:09.055946+0000","last_became_active":"2024-05-27T08:35:07.048540+0000","last_became_peered":"2024-05-27T08:35:07.048540+0000","last_unstale":"2024-05-27T08:35:09.055946+0000","last_undegraded":"2024-05-27T08:35:09.055946+0000","last_fullsized":"2024-05-27T08:35:09.055946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:56.219362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052611+0000","last_change":"2024-05-27T08:35:07.049775+0000","last_active":"2024-05-27T08:35:09.052611+0000","last_peered":"2024-05-27T08:35:09.052611+0000","last_clean":"2024-05-27T08:35:09.052611+0000","last_became_active":"2024-05-27T08:35:07.049466+0000","last_became_peered":"2024-05-27T08:35:07.049466+0000","last_unstale":"2024-05-27T08:35:09.052611+0000","last_undegraded":"2024-05-27T08:35:09.052611+0000","last_fullsized":"2024-05-27T08:35:09.052611+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:18:30.394390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050109+0000","last_change":"2024-05-27T08:35:07.050870+0000","last_active":"2024-05-27T08:35:09.050109+0000","last_peered":"2024-05-27T08:35:09.050109+0000","last_clean":"2024-05-27T08:35:09.050109+0000","last_became_active":"2024-05-27T08:35:07.050527+0000","last_became_peered":"2024-05-27T08:35:07.050527+0000","last_unstale":"2024-05-27T08:35:09.050109+0000","last_undegraded":"2024-05-27T08:35:09.050109+0000","last_fullsized":"2024-05-27T08:35:09.050109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:56:41.477053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.043671+0000","last_change":"2024-05-27T08:35:07.052955+0000","last_active":"2024-05-27T08:35:09.043671+0000","last_peered":"2024-05-27T08:35:09.043671+0000","last_clean":"2024-05-27T08:35:09.043671+0000","last_became_active":"2024-05-27T08:35:07.052500+0000","last_became_peered":"2024-05-27T08:35:07.052500+0000","last_unstale":"2024-05-27T08:35:09.043671+0000","last_undegraded":"2024-05-27T08:35:09.043671+0000","last_fullsized":"2024-05-27T08:35:09.043671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:25.093408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051940+0000","last_change":"2024-05-27T08:35:07.044866+0000","last_active":"2024-05-27T08:35:09.051940+0000","last_peered":"2024-05-27T08:35:09.051940+0000","last_clean":"2024-05-27T08:35:09.051940+0000","last_became_active":"2024-05-27T08:35:07.044554+0000","last_became_peered":"2024-05-27T08:35:07.044554+0000","last_unstale":"2024-05-27T08:35:09.051940+0000","last_undegraded":"2024-05-27T08:35:09.051940+0000","last_fullsized":"2024-05-27T08:35:09.051940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:45.624970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054503+0000","last_change":"2024-05-27T08:35:07.043985+0000","last_active":"2024-05-27T08:35:09.054503+0000","last_peered":"2024-05-27T08:35:09.054503+0000","last_clean":"2024-05-27T08:35:09.054503+0000","last_became_active":"2024-05-27T08:35:07.043694+0000","last_became_peered":"2024-05-27T08:35:07.043694+0000","last_unstale":"2024-05-27T08:35:09.054503+0000","last_undegraded":"2024-05-27T08:35:09.054503+0000","last_fullsized":"2024-05-27T08:35:09.054503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:22:49.796670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054153+0000","last_change":"2024-05-27T08:35:07.045542+0000","last_active":"2024-05-27T08:35:09.054153+0000","last_peered":"2024-05-27T08:35:09.054153+0000","last_clean":"2024-05-27T08:35:09.054153+0000","last_became_active":"2024-05-27T08:35:07.045209+0000","last_became_peered":"2024-05-27T08:35:07.045209+0000","last_unstale":"2024-05-27T08:35:09.054153+0000","last_undegraded":"2024-05-27T08:35:09.054153+0000","last_fullsized":"2024-05-27T08:35:09.054153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:57.733089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054230+0000","last_change":"2024-05-27T08:35:07.047193+0000","last_active":"2024-05-27T08:35:09.054230+0000","last_peered":"2024-05-27T08:35:09.054230+0000","last_clean":"2024-05-27T08:35:09.054230+0000","last_became_active":"2024-05-27T08:35:07.046862+0000","last_became_peered":"2024-05-27T08:35:07.046862+0000","last_unstale":"2024-05-27T08:35:09.054230+0000","last_undegraded":"2024-05-27T08:35:09.054230+0000","last_fullsized":"2024-05-27T08:35:09.054230+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:21.288642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050766+0000","last_change":"2024-05-27T08:35:07.045452+0000","last_active":"2024-05-27T08:35:09.050766+0000","last_peered":"2024-05-27T08:35:09.050766+0000","last_clean":"2024-05-27T08:35:09.050766+0000","last_became_active":"2024-05-27T08:35:07.044783+0000","last_became_peered":"2024-05-27T08:35:07.044783+0000","last_unstale":"2024-05-27T08:35:09.050766+0000","last_undegraded":"2024-05-27T08:35:09.050766+0000","last_fullsized":"2024-05-27T08:35:09.050766+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:03.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042696+0000","last_change":"2024-05-27T08:35:07.042864+0000","last_active":"2024-05-27T08:35:09.042696+0000","last_peered":"2024-05-27T08:35:09.042696+0000","last_clean":"2024-05-27T08:35:09.042696+0000","last_became_active":"2024-05-27T08:35:07.042523+0000","last_became_peered":"2024-05-27T08:35:07.042523+0000","last_unstale":"2024-05-27T08:35:09.042696+0000","last_undegraded":"2024-05-27T08:35:09.042696+0000","last_fullsized":"2024-05-27T08:35:09.042696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:34.275977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042875+0000","last_change":"2024-05-27T08:35:07.042667+0000","last_active":"2024-05-27T08:35:09.042875+0000","last_peered":"2024-05-27T08:35:09.042875+0000","last_clean":"2024-05-27T08:35:09.042875+0000","last_became_active":"2024-05-27T08:35:07.042090+0000","last_became_peered":"2024-05-27T08:35:07.042090+0000","last_unstale":"2024-05-27T08:35:09.042875+0000","last_undegraded":"2024-05-27T08:35:09.042875+0000","last_fullsized":"2024-05-27T08:35:09.042875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:28.764641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":59,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:46.307647+0000","last_change":"2024-05-27T08:35:04.021045+0000","last_active":"2024-05-27T08:35:46.307647+0000","last_peered":"2024-05-27T08:35:46.307647+0000","last_clean":"2024-05-27T08:35:46.307647+0000","last_became_active":"2024-05-27T08:35:04.020721+0000","last_became_peered":"2024-05-27T08:35:04.020721+0000","last_unstale":"2024-05-27T08:35:46.307647+0000","last_undegraded":"2024-05-27T08:35:46.307647+0000","last_fullsized":"2024-05-27T08:35:46.307647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:42:02.954131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050163+0000","last_change":"2024-05-27T08:35:07.049605+0000","last_active":"2024-05-27T08:35:09.050163+0000","last_peered":"2024-05-27T08:35:09.050163+0000","last_clean":"2024-05-27T08:35:09.050163+0000","last_became_active":"2024-05-27T08:35:07.049307+0000","last_became_peered":"2024-05-27T08:35:07.049307+0000","last_unstale":"2024-05-27T08:35:09.050163+0000","last_undegraded":"2024-05-27T08:35:09.050163+0000","last_fullsized":"2024-05-27T08:35:09.050163+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:37:19.286030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054638+0000","last_change":"2024-05-27T08:35:04.245013+0000","last_active":"2024-05-27T08:35:09.054638+0000","last_peered":"2024-05-27T08:35:09.054638+0000","last_clean":"2024-05-27T08:35:09.054638+0000","last_became_active":"2024-05-27T08:35:04.244629+0000","last_became_peered":"2024-05-27T08:35:04.244629+0000","last_unstale":"2024-05-27T08:35:09.054638+0000","last_undegraded":"2024-05-27T08:35:09.054638+0000","last_fullsized":"2024-05-27T08:35:09.054638+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:41:43.747592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055065+0000","last_change":"2024-05-27T08:35:07.041776+0000","last_active":"2024-05-27T08:35:09.055065+0000","last_peered":"2024-05-27T08:35:09.055065+0000","last_clean":"2024-05-27T08:35:09.055065+0000","last_became_active":"2024-05-27T08:35:07.041400+0000","last_became_peered":"2024-05-27T08:35:07.041400+0000","last_unstale":"2024-05-27T08:35:09.055065+0000","last_undegraded":"2024-05-27T08:35:09.055065+0000","last_fullsized":"2024-05-27T08:35:09.055065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:10.091310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:24.269937+0000","last_change":"2024-05-27T08:35:04.023072+0000","last_active":"2024-05-27T08:35:24.269937+0000","last_peered":"2024-05-27T08:35:24.269937+0000","last_clean":"2024-05-27T08:35:24.269937+0000","last_became_active":"2024-05-27T08:35:04.022232+0000","last_became_peered":"2024-05-27T08:35:04.022232+0000","last_unstale":"2024-05-27T08:35:24.269937+0000","last_undegraded":"2024-05-27T08:35:24.269937+0000","last_fullsized":"2024-05-27T08:35:24.269937+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:49:57.240621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055373+0000","last_change":"2024-05-27T08:35:07.044727+0000","last_active":"2024-05-27T08:35:09.055373+0000","last_peered":"2024-05-27T08:35:09.055373+0000","last_clean":"2024-05-27T08:35:09.055373+0000","last_became_active":"2024-05-27T08:35:07.044352+0000","last_became_peered":"2024-05-27T08:35:07.044352+0000","last_unstale":"2024-05-27T08:35:09.055373+0000","last_undegraded":"2024-05-27T08:35:09.055373+0000","last_fullsized":"2024-05-27T08:35:09.055373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:52:43.473606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.275112+0000","last_change":"2024-05-27T08:35:04.019179+0000","last_active":"2024-05-27T08:35:23.275112+0000","last_peered":"2024-05-27T08:35:23.275112+0000","last_clean":"2024-05-27T08:35:23.275112+0000","last_became_active":"2024-05-27T08:35:04.018726+0000","last_became_peered":"2024-05-27T08:35:04.018726+0000","last_unstale":"2024-05-27T08:35:23.275112+0000","last_undegraded":"2024-05-27T08:35:23.275112+0000","last_fullsized":"2024-05-27T08:35:23.275112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:37:47.744001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.057487+0000","last_change":"2024-05-27T08:35:07.055562+0000","last_active":"2024-05-27T08:35:09.057487+0000","last_peered":"2024-05-27T08:35:09.057487+0000","last_clean":"2024-05-27T08:35:09.057487+0000","last_became_active":"2024-05-27T08:35:07.055209+0000","last_became_peered":"2024-05-27T08:35:07.055209+0000","last_unstale":"2024-05-27T08:35:09.057487+0000","last_undegraded":"2024-05-27T08:35:09.057487+0000","last_fullsized":"2024-05-27T08:35:09.057487+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:04.920534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:47.310934+0000","last_change":"2024-05-27T08:35:04.022778+0000","last_active":"2024-05-27T08:35:47.310934+0000","last_peered":"2024-05-27T08:35:47.310934+0000","last_clean":"2024-05-27T08:35:47.310934+0000","last_became_active":"2024-05-27T08:35:04.022311+0000","last_became_peered":"2024-05-27T08:35:04.022311+0000","last_unstale":"2024-05-27T08:35:47.310934+0000","last_undegraded":"2024-05-27T08:35:47.310934+0000","last_fullsized":"2024-05-27T08:35:47.310934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:48.174774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050700+0000","last_change":"2024-05-27T08:35:07.040182+0000","last_active":"2024-05-27T08:35:09.050700+0000","last_peered":"2024-05-27T08:35:09.050700+0000","last_clean":"2024-05-27T08:35:09.050700+0000","last_became_active":"2024-05-27T08:35:07.039800+0000","last_became_peered":"2024-05-27T08:35:07.039800+0000","last_unstale":"2024-05-27T08:35:09.050700+0000","last_undegraded":"2024-05-27T08:35:09.050700+0000","last_fullsized":"2024-05-27T08:35:09.050700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:53:53.470284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:22.258772+0000","last_change":"2024-05-27T08:35:04.023757+0000","last_active":"2024-05-27T08:35:22.258772+0000","last_peered":"2024-05-27T08:35:22.258772+0000","last_clean":"2024-05-27T08:35:22.258772+0000","last_became_active":"2024-05-27T08:35:04.023428+0000","last_became_peered":"2024-05-27T08:35:04.023428+0000","last_unstale":"2024-05-27T08:35:22.258772+0000","last_undegraded":"2024-05-27T08:35:22.258772+0000","last_fullsized":"2024-05-27T08:35:22.258772+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:45.669006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.056452+0000","last_change":"2024-05-27T08:35:07.051854+0000","last_active":"2024-05-27T08:35:09.056452+0000","last_peered":"2024-05-27T08:35:09.056452+0000","last_clean":"2024-05-27T08:35:09.056452+0000","last_became_active":"2024-05-27T08:35:07.051468+0000","last_became_peered":"2024-05-27T08:35:07.051468+0000","last_unstale":"2024-05-27T08:35:09.056452+0000","last_undegraded":"2024-05-27T08:35:09.056452+0000","last_fullsized":"2024-05-27T08:35:09.056452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:20:58.998776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:21.253807+0000","last_change":"2024-05-27T08:35:04.015581+0000","last_active":"2024-05-27T08:35:21.253807+0000","last_peered":"2024-05-27T08:35:21.253807+0000","last_clean":"2024-05-27T08:35:21.253807+0000","last_became_active":"2024-05-27T08:35:04.015215+0000","last_became_peered":"2024-05-27T08:35:04.015215+0000","last_unstale":"2024-05-27T08:35:21.253807+0000","last_undegraded":"2024-05-27T08:35:21.253807+0000","last_fullsized":"2024-05-27T08:35:21.253807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:21:51.942008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050014+0000","last_change":"2024-05-27T08:35:07.049798+0000","last_active":"2024-05-27T08:35:09.050014+0000","last_peered":"2024-05-27T08:35:09.050014+0000","last_clean":"2024-05-27T08:35:09.050014+0000","last_became_active":"2024-05-27T08:35:07.048686+0000","last_became_peered":"2024-05-27T08:35:07.048686+0000","last_unstale":"2024-05-27T08:35:09.050014+0000","last_undegraded":"2024-05-27T08:35:09.050014+0000","last_fullsized":"2024-05-27T08:35:09.050014+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:39:23.817581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.274337+0000","last_change":"2024-05-27T08:35:04.023826+0000","last_active":"2024-05-27T08:35:23.274337+0000","last_peered":"2024-05-27T08:35:23.274337+0000","last_clean":"2024-05-27T08:35:23.274337+0000","last_became_active":"2024-05-27T08:35:04.023410+0000","last_became_peered":"2024-05-27T08:35:04.023410+0000","last_unstale":"2024-05-27T08:35:23.274337+0000","last_undegraded":"2024-05-27T08:35:23.274337+0000","last_fullsized":"2024-05-27T08:35:23.274337+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:38:47.855186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042256+0000","last_change":"2024-05-27T08:35:07.038810+0000","last_active":"2024-05-27T08:35:09.042256+0000","last_peered":"2024-05-27T08:35:09.042256+0000","last_clean":"2024-05-27T08:35:09.042256+0000","last_became_active":"2024-05-27T08:35:07.038480+0000","last_became_peered":"2024-05-27T08:35:07.038480+0000","last_unstale":"2024-05-27T08:35:09.042256+0000","last_undegraded":"2024-05-27T08:35:09.042256+0000","last_fullsized":"2024-05-27T08:35:09.042256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:14.429081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":78,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:49.313033+0000","last_change":"2024-05-27T08:35:04.026934+0000","last_active":"2024-05-27T08:35:49.313033+0000","last_peered":"2024-05-27T08:35:49.313033+0000","last_clean":"2024-05-27T08:35:49.313033+0000","last_became_active":"2024-05-27T08:35:04.026577+0000","last_became_peered":"2024-05-27T08:35:04.026577+0000","last_unstale":"2024-05-27T08:35:49.313033+0000","last_undegraded":"2024-05-27T08:35:49.313033+0000","last_fullsized":"2024-05-27T08:35:49.313033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:59:56.067856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055396+0000","last_change":"2024-05-27T08:35:07.043840+0000","last_active":"2024-05-27T08:35:09.055396+0000","last_peered":"2024-05-27T08:35:09.055396+0000","last_clean":"2024-05-27T08:35:09.055396+0000","last_became_active":"2024-05-27T08:35:07.043224+0000","last_became_peered":"2024-05-27T08:35:07.043224+0000","last_unstale":"2024-05-27T08:35:09.055396+0000","last_undegraded":"2024-05-27T08:35:09.055396+0000","last_fullsized":"2024-05-27T08:35:09.055396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:20.212038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.274717+0000","last_change":"2024-05-27T08:35:04.022602+0000","last_active":"2024-05-27T08:35:23.274717+0000","last_peered":"2024-05-27T08:35:23.274717+0000","last_clean":"2024-05-27T08:35:23.274717+0000","last_became_active":"2024-05-27T08:35:04.022203+0000","last_became_peered":"2024-05-27T08:35:04.022203+0000","last_unstale":"2024-05-27T08:35:23.274717+0000","last_undegraded":"2024-05-27T08:35:23.274717+0000","last_fullsized":"2024-05-27T08:35:23.274717+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:51.490696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054951+0000","last_change":"2024-05-27T08:35:07.047257+0000","last_active":"2024-05-27T08:35:09.054951+0000","last_peered":"2024-05-27T08:35:09.054951+0000","last_clean":"2024-05-27T08:35:09.054951+0000","last_became_active":"2024-05-27T08:35:07.046639+0000","last_became_peered":"2024-05-27T08:35:07.046639+0000","last_unstale":"2024-05-27T08:35:09.054951+0000","last_undegraded":"2024-05-27T08:35:09.054951+0000","last_fullsized":"2024-05-27T08:35:09.054951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:04.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:22.259517+0000","last_change":"2024-05-27T08:35:04.245295+0000","last_active":"2024-05-27T08:35:22.259517+0000","last_peered":"2024-05-27T08:35:22.259517+0000","last_clean":"2024-05-27T08:35:22.259517+0000","last_became_active":"2024-05-27T08:35:04.244927+0000","last_became_peered":"2024-05-27T08:35:04.244927+0000","last_unstale":"2024-05-27T08:35:22.259517+0000","last_undegraded":"2024-05-27T08:35:22.259517+0000","last_fullsized":"2024-05-27T08:35:22.259517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:58.893480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054980+0000","last_change":"2024-05-27T08:35:07.048872+0000","last_active":"2024-05-27T08:35:09.054980+0000","last_peered":"2024-05-27T08:35:09.054980+0000","last_clean":"2024-05-27T08:35:09.054980+0000","last_became_active":"2024-05-27T08:35:07.048488+0000","last_became_peered":"2024-05-27T08:35:07.048488+0000","last_unstale":"2024-05-27T08:35:09.054980+0000","last_undegraded":"2024-05-27T08:35:09.054980+0000","last_fullsized":"2024-05-27T08:35:09.054980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:31.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":114,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:49.318776+0000","last_change":"2024-05-27T08:35:04.024718+0000","last_active":"2024-05-27T08:35:49.318776+0000","last_peered":"2024-05-27T08:35:49.318776+0000","last_clean":"2024-05-27T08:35:49.318776+0000","last_became_active":"2024-05-27T08:35:04.024155+0000","last_became_peered":"2024-05-27T08:35:04.024155+0000","last_unstale":"2024-05-27T08:35:49.318776+0000","last_undegraded":"2024-05-27T08:35:49.318776+0000","last_fullsized":"2024-05-27T08:35:49.318776+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:22.573337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":88,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052984+0000","last_change":"2024-05-27T08:35:07.051342+0000","last_active":"2024-05-27T08:35:09.052984+0000","last_peered":"2024-05-27T08:35:09.052984+0000","last_clean":"2024-05-27T08:35:09.052984+0000","last_became_active":"2024-05-27T08:35:07.051030+0000","last_became_peered":"2024-05-27T08:35:07.051030+0000","last_unstale":"2024-05-27T08:35:09.052984+0000","last_undegraded":"2024-05-27T08:35:09.052984+0000","last_fullsized":"2024-05-27T08:35:09.052984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:48.245036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:24.275752+0000","last_change":"2024-05-27T08:35:04.023717+0000","last_active":"2024-05-27T08:35:24.275752+0000","last_peered":"2024-05-27T08:35:24.275752+0000","last_clean":"2024-05-27T08:35:24.275752+0000","last_became_active":"2024-05-27T08:35:04.022017+0000","last_became_peered":"2024-05-27T08:35:04.022017+0000","last_unstale":"2024-05-27T08:35:24.275752+0000","last_undegraded":"2024-05-27T08:35:24.275752+0000","last_fullsized":"2024-05-27T08:35:24.275752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:55.834307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050665+0000","last_change":"2024-05-27T08:35:07.043980+0000","last_active":"2024-05-27T08:35:09.050665+0000","last_peered":"2024-05-27T08:35:09.050665+0000","last_clean":"2024-05-27T08:35:09.050665+0000","last_became_active":"2024-05-27T08:35:07.043672+0000","last_became_peered":"2024-05-27T08:35:07.043672+0000","last_unstale":"2024-05-27T08:35:09.050665+0000","last_undegraded":"2024-05-27T08:35:09.050665+0000","last_fullsized":"2024-05-27T08:35:09.050665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:03:30.243966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273424+0000","last_change":"2024-05-27T08:35:04.022731+0000","last_active":"2024-05-27T08:35:23.273424+0000","last_peered":"2024-05-27T08:35:23.273424+0000","last_clean":"2024-05-27T08:35:23.273424+0000","last_became_active":"2024-05-27T08:35:04.022349+0000","last_became_peered":"2024-05-27T08:35:04.022349+0000","last_unstale":"2024-05-27T08:35:23.273424+0000","last_undegraded":"2024-05-27T08:35:23.273424+0000","last_fullsized":"2024-05-27T08:35:23.273424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:38.651520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049580+0000","last_change":"2024-05-27T08:35:07.052623+0000","last_active":"2024-05-27T08:35:09.049580+0000","last_peered":"2024-05-27T08:35:09.049580+0000","last_clean":"2024-05-27T08:35:09.049580+0000","last_became_active":"2024-05-27T08:35:07.052313+0000","last_became_peered":"2024-05-27T08:35:07.052313+0000","last_unstale":"2024-05-27T08:35:09.049580+0000","last_undegraded":"2024-05-27T08:35:09.049580+0000","last_fullsized":"2024-05-27T08:35:09.049580+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:24.612523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":55,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:47.316582+0000","last_change":"2024-05-27T08:35:04.023756+0000","last_active":"2024-05-27T08:35:47.316582+0000","last_peered":"2024-05-27T08:35:47.316582+0000","last_clean":"2024-05-27T08:35:47.316582+0000","last_became_active":"2024-05-27T08:35:04.023401+0000","last_became_peered":"2024-05-27T08:35:04.023401+0000","last_unstale":"2024-05-27T08:35:47.316582+0000","last_undegraded":"2024-05-27T08:35:47.316582+0000","last_fullsized":"2024-05-27T08:35:47.316582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:34.887246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049269+0000","last_change":"2024-05-27T08:35:07.050720+0000","last_active":"2024-05-27T08:35:09.049269+0000","last_peered":"2024-05-27T08:35:09.049269+0000","last_clean":"2024-05-27T08:35:09.049269+0000","last_became_active":"2024-05-27T08:35:07.050135+0000","last_became_peered":"2024-05-27T08:35:07.050135+0000","last_unstale":"2024-05-27T08:35:09.049269+0000","last_undegraded":"2024-05-27T08:35:09.049269+0000","last_fullsized":"2024-05-27T08:35:09.049269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:40.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273117+0000","last_change":"2024-05-27T08:35:04.245264+0000","last_active":"2024-05-27T08:35:23.273117+0000","last_peered":"2024-05-27T08:35:23.273117+0000","last_clean":"2024-05-27T08:35:23.273117+0000","last_became_active":"2024-05-27T08:35:04.244776+0000","last_became_peered":"2024-05-27T08:35:04.244776+0000","last_unstale":"2024-05-27T08:35:23.273117+0000","last_undegraded":"2024-05-27T08:35:23.273117+0000","last_fullsized":"2024-05-27T08:35:23.273117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:27:55.370081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.055345+0000","last_change":"2024-05-27T08:35:07.050504+0000","last_active":"2024-05-27T08:35:09.055345+0000","last_peered":"2024-05-27T08:35:09.055345+0000","last_clean":"2024-05-27T08:35:09.055345+0000","last_became_active":"2024-05-27T08:35:07.049368+0000","last_became_peered":"2024-05-27T08:35:07.049368+0000","last_unstale":"2024-05-27T08:35:09.055345+0000","last_undegraded":"2024-05-27T08:35:09.055345+0000","last_fullsized":"2024-05-27T08:35:09.055345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:52.996313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:22.248329+0000","last_change":"2024-05-27T08:35:04.023924+0000","last_active":"2024-05-27T08:35:22.248329+0000","last_peered":"2024-05-27T08:35:22.248329+0000","last_clean":"2024-05-27T08:35:22.248329+0000","last_became_active":"2024-05-27T08:35:04.023548+0000","last_became_peered":"2024-05-27T08:35:04.023548+0000","last_unstale":"2024-05-27T08:35:22.248329+0000","last_undegraded":"2024-05-27T08:35:22.248329+0000","last_fullsized":"2024-05-27T08:35:22.248329+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:50.441490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050280+0000","last_change":"2024-05-27T08:35:07.051509+0000","last_active":"2024-05-27T08:35:09.050280+0000","last_peered":"2024-05-27T08:35:09.050280+0000","last_clean":"2024-05-27T08:35:09.050280+0000","last_became_active":"2024-05-27T08:35:07.051184+0000","last_became_peered":"2024-05-27T08:35:07.051184+0000","last_unstale":"2024-05-27T08:35:09.050280+0000","last_undegraded":"2024-05-27T08:35:09.050280+0000","last_fullsized":"2024-05-27T08:35:09.050280+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:17:53.033093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042465+0000","last_change":"2024-05-27T08:35:04.245702+0000","last_active":"2024-05-27T08:35:09.042465+0000","last_peered":"2024-05-27T08:35:09.042465+0000","last_clean":"2024-05-27T08:35:09.042465+0000","last_became_active":"2024-05-27T08:35:04.244624+0000","last_became_peered":"2024-05-27T08:35:04.244624+0000","last_unstale":"2024-05-27T08:35:09.042465+0000","last_undegraded":"2024-05-27T08:35:09.042465+0000","last_fullsized":"2024-05-27T08:35:09.042465+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:43:11.055147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.449778+0000","last_change":"2024-05-27T08:35:07.039182+0000","last_active":"2024-05-27T08:35:09.449778+0000","last_peered":"2024-05-27T08:35:09.449778+0000","last_clean":"2024-05-27T08:35:09.449778+0000","last_became_active":"2024-05-27T08:35:07.038793+0000","last_became_peered":"2024-05-27T08:35:07.038793+0000","last_unstale":"2024-05-27T08:35:09.449778+0000","last_undegraded":"2024-05-27T08:35:09.449778+0000","last_fullsized":"2024-05-27T08:35:09.449778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:03.839681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:45.306821+0000","last_change":"2024-05-27T08:35:04.023194+0000","last_active":"2024-05-27T08:35:45.306821+0000","last_peered":"2024-05-27T08:35:45.306821+0000","last_clean":"2024-05-27T08:35:45.306821+0000","last_became_active":"2024-05-27T08:35:04.022823+0000","last_became_peered":"2024-05-27T08:35:04.022823+0000","last_unstale":"2024-05-27T08:35:45.306821+0000","last_undegraded":"2024-05-27T08:35:45.306821+0000","last_fullsized":"2024-05-27T08:35:45.306821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:44:28.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051863+0000","last_change":"2024-05-27T08:35:07.050717+0000","last_active":"2024-05-27T08:35:09.051863+0000","last_peered":"2024-05-27T08:35:09.051863+0000","last_clean":"2024-05-27T08:35:09.051863+0000","last_became_active":"2024-05-27T08:35:07.050413+0000","last_became_peered":"2024-05-27T08:35:07.050413+0000","last_unstale":"2024-05-27T08:35:09.051863+0000","last_undegraded":"2024-05-27T08:35:09.051863+0000","last_fullsized":"2024-05-27T08:35:09.051863+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:00:27.693141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.058361+0000","last_change":"2024-05-27T08:35:07.053614+0000","last_active":"2024-05-27T08:35:09.058361+0000","last_peered":"2024-05-27T08:35:09.058361+0000","last_clean":"2024-05-27T08:35:09.058361+0000","last_became_active":"2024-05-27T08:35:07.053253+0000","last_became_peered":"2024-05-27T08:35:07.053253+0000","last_unstale":"2024-05-27T08:35:09.058361+0000","last_undegraded":"2024-05-27T08:35:09.058361+0000","last_fullsized":"2024-05-27T08:35:09.058361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:15:08.985984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":52,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:46.314980+0000","last_change":"2024-05-27T08:35:04.246500+0000","last_active":"2024-05-27T08:35:46.314980+0000","last_peered":"2024-05-27T08:35:46.314980+0000","last_clean":"2024-05-27T08:35:46.314980+0000","last_became_active":"2024-05-27T08:35:04.244575+0000","last_became_peered":"2024-05-27T08:35:04.244575+0000","last_unstale":"2024-05-27T08:35:46.314980+0000","last_undegraded":"2024-05-27T08:35:46.314980+0000","last_fullsized":"2024-05-27T08:35:46.314980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:24:10.818057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050867+0000","last_change":"2024-05-27T08:35:07.048488+0000","last_active":"2024-05-27T08:35:09.050867+0000","last_peered":"2024-05-27T08:35:09.050867+0000","last_clean":"2024-05-27T08:35:09.050867+0000","last_became_active":"2024-05-27T08:35:07.048172+0000","last_became_peered":"2024-05-27T08:35:07.048172+0000","last_unstale":"2024-05-27T08:35:09.050867+0000","last_undegraded":"2024-05-27T08:35:09.050867+0000","last_fullsized":"2024-05-27T08:35:09.050867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:01.632944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:24.277934+0000","last_change":"2024-05-27T08:35:04.022807+0000","last_active":"2024-05-27T08:35:24.277934+0000","last_peered":"2024-05-27T08:35:24.277934+0000","last_clean":"2024-05-27T08:35:24.277934+0000","last_became_active":"2024-05-27T08:35:04.021198+0000","last_became_peered":"2024-05-27T08:35:04.021198+0000","last_unstale":"2024-05-27T08:35:24.277934+0000","last_undegraded":"2024-05-27T08:35:24.277934+0000","last_fullsized":"2024-05-27T08:35:24.277934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:42:10.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":96,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049544+0000","last_change":"2024-05-27T08:33:45.683600+0000","last_active":"2024-05-27T08:35:09.049544+0000","last_peered":"2024-05-27T08:35:09.049544+0000","last_clean":"2024-05-27T08:35:09.049544+0000","last_became_active":"2024-05-27T08:33:45.677989+0000","last_became_peered":"2024-05-27T08:33:45.677989+0000","last_unstale":"2024-05-27T08:35:09.049544+0000","last_undegraded":"2024-05-27T08:35:09.049544+0000","last_fullsized":"2024-05-27T08:35:09.049544+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_clean_scrub_stamp":"2024-05-27T08:30:25.507727+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:48:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051232+0000","last_change":"2024-05-27T08:35:07.045879+0000","last_active":"2024-05-27T08:35:09.051232+0000","last_peered":"2024-05-27T08:35:09.051232+0000","last_clean":"2024-05-27T08:35:09.051232+0000","last_became_active":"2024-05-27T08:35:07.045522+0000","last_became_peered":"2024-05-27T08:35:07.045522+0000","last_unstale":"2024-05-27T08:35:09.051232+0000","last_undegraded":"2024-05-27T08:35:09.051232+0000","last_fullsized":"2024-05-27T08:35:09.051232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:30.393769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:21.253160+0000","last_change":"2024-05-27T08:35:04.022419+0000","last_active":"2024-05-27T08:35:21.253160+0000","last_peered":"2024-05-27T08:35:21.253160+0000","last_clean":"2024-05-27T08:35:21.253160+0000","last_became_active":"2024-05-27T08:35:04.022039+0000","last_became_peered":"2024-05-27T08:35:04.022039+0000","last_unstale":"2024-05-27T08:35:21.253160+0000","last_undegraded":"2024-05-27T08:35:21.253160+0000","last_fullsized":"2024-05-27T08:35:21.253160+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:31.260596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273484+0000","last_change":"2024-05-27T08:35:04.245214+0000","last_active":"2024-05-27T08:35:23.273484+0000","last_peered":"2024-05-27T08:35:23.273484+0000","last_clean":"2024-05-27T08:35:23.273484+0000","last_became_active":"2024-05-27T08:35:04.244768+0000","last_became_peered":"2024-05-27T08:35:04.244768+0000","last_unstale":"2024-05-27T08:35:23.273484+0000","last_undegraded":"2024-05-27T08:35:23.273484+0000","last_fullsized":"2024-05-27T08:35:23.273484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:34.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052346+0000","last_change":"2024-05-27T08:35:07.042721+0000","last_active":"2024-05-27T08:35:09.052346+0000","last_peered":"2024-05-27T08:35:09.052346+0000","last_clean":"2024-05-27T08:35:09.052346+0000","last_became_active":"2024-05-27T08:35:07.042435+0000","last_became_peered":"2024-05-27T08:35:07.042435+0000","last_unstale":"2024-05-27T08:35:09.052346+0000","last_undegraded":"2024-05-27T08:35:09.052346+0000","last_fullsized":"2024-05-27T08:35:09.052346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:49.186753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:43.052261+0000","last_change":"2024-05-27T08:35:04.017995+0000","last_active":"2024-05-27T08:35:43.052261+0000","last_peered":"2024-05-27T08:35:43.052261+0000","last_clean":"2024-05-27T08:35:43.052261+0000","last_became_active":"2024-05-27T08:35:04.017675+0000","last_became_peered":"2024-05-27T08:35:04.017675+0000","last_unstale":"2024-05-27T08:35:43.052261+0000","last_undegraded":"2024-05-27T08:35:43.052261+0000","last_fullsized":"2024-05-27T08:35:43.052261+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:23:15.452624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.051045+0000","last_change":"2024-05-27T08:35:07.046553+0000","last_active":"2024-05-27T08:35:09.051045+0000","last_peered":"2024-05-27T08:35:09.051045+0000","last_clean":"2024-05-27T08:35:09.051045+0000","last_became_active":"2024-05-27T08:35:07.046229+0000","last_became_peered":"2024-05-27T08:35:07.046229+0000","last_unstale":"2024-05-27T08:35:09.051045+0000","last_undegraded":"2024-05-27T08:35:09.051045+0000","last_fullsized":"2024-05-27T08:35:09.051045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:21.024307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273336+0000","last_change":"2024-05-27T08:35:04.262439+0000","last_active":"2024-05-27T08:35:23.273336+0000","last_peered":"2024-05-27T08:35:23.273336+0000","last_clean":"2024-05-27T08:35:23.273336+0000","last_became_active":"2024-05-27T08:35:04.261958+0000","last_became_peered":"2024-05-27T08:35:04.261958+0000","last_unstale":"2024-05-27T08:35:23.273336+0000","last_undegraded":"2024-05-27T08:35:23.273336+0000","last_fullsized":"2024-05-27T08:35:23.273336+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:11:05.763068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054546+0000","last_change":"2024-05-27T08:35:07.042307+0000","last_active":"2024-05-27T08:35:09.054546+0000","last_peered":"2024-05-27T08:35:09.054546+0000","last_clean":"2024-05-27T08:35:09.054546+0000","last_became_active":"2024-05-27T08:35:07.042001+0000","last_became_peered":"2024-05-27T08:35:07.042001+0000","last_unstale":"2024-05-27T08:35:09.054546+0000","last_undegraded":"2024-05-27T08:35:09.054546+0000","last_fullsized":"2024-05-27T08:35:09.054546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:51.030157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:20.240835+0000","last_change":"2024-05-27T08:35:04.245436+0000","last_active":"2024-05-27T08:35:20.240835+0000","last_peered":"2024-05-27T08:35:20.240835+0000","last_clean":"2024-05-27T08:35:20.240835+0000","last_became_active":"2024-05-27T08:35:04.245039+0000","last_became_peered":"2024-05-27T08:35:04.245039+0000","last_unstale":"2024-05-27T08:35:20.240835+0000","last_undegraded":"2024-05-27T08:35:20.240835+0000","last_fullsized":"2024-05-27T08:35:20.240835+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:10:11.756864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050358+0000","last_change":"2024-05-27T08:35:07.051524+0000","last_active":"2024-05-27T08:35:09.050358+0000","last_peered":"2024-05-27T08:35:09.050358+0000","last_clean":"2024-05-27T08:35:09.050358+0000","last_became_active":"2024-05-27T08:35:07.050676+0000","last_became_peered":"2024-05-27T08:35:07.050676+0000","last_unstale":"2024-05-27T08:35:09.050358+0000","last_undegraded":"2024-05-27T08:35:09.050358+0000","last_fullsized":"2024-05-27T08:35:09.050358+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:06.818977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273411+0000","last_change":"2024-05-27T08:35:04.022064+0000","last_active":"2024-05-27T08:35:23.273411+0000","last_peered":"2024-05-27T08:35:23.273411+0000","last_clean":"2024-05-27T08:35:23.273411+0000","last_became_active":"2024-05-27T08:35:04.021691+0000","last_became_peered":"2024-05-27T08:35:04.021691+0000","last_unstale":"2024-05-27T08:35:23.273411+0000","last_undegraded":"2024-05-27T08:35:23.273411+0000","last_fullsized":"2024-05-27T08:35:23.273411+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:54.557756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050151+0000","last_change":"2024-05-27T08:35:07.049666+0000","last_active":"2024-05-27T08:35:09.050151+0000","last_peered":"2024-05-27T08:35:09.050151+0000","last_clean":"2024-05-27T08:35:09.050151+0000","last_became_active":"2024-05-27T08:35:07.049312+0000","last_became_peered":"2024-05-27T08:35:07.049312+0000","last_unstale":"2024-05-27T08:35:09.050151+0000","last_undegraded":"2024-05-27T08:35:09.050151+0000","last_fullsized":"2024-05-27T08:35:09.050151+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:37:53.704582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:22.259907+0000","last_change":"2024-05-27T08:35:04.018457+0000","last_active":"2024-05-27T08:35:22.259907+0000","last_peered":"2024-05-27T08:35:22.259907+0000","last_clean":"2024-05-27T08:35:22.259907+0000","last_became_active":"2024-05-27T08:35:04.018008+0000","last_became_peered":"2024-05-27T08:35:04.018008+0000","last_unstale":"2024-05-27T08:35:22.259907+0000","last_undegraded":"2024-05-27T08:35:22.259907+0000","last_fullsized":"2024-05-27T08:35:22.259907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:08:13.228725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050512+0000","last_change":"2024-05-27T08:35:07.047040+0000","last_active":"2024-05-27T08:35:09.050512+0000","last_peered":"2024-05-27T08:35:09.050512+0000","last_clean":"2024-05-27T08:35:09.050512+0000","last_became_active":"2024-05-27T08:35:07.046731+0000","last_became_peered":"2024-05-27T08:35:07.046731+0000","last_unstale":"2024-05-27T08:35:09.050512+0000","last_undegraded":"2024-05-27T08:35:09.050512+0000","last_fullsized":"2024-05-27T08:35:09.050512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:25.368349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:20.241434+0000","last_change":"2024-05-27T08:35:04.022804+0000","last_active":"2024-05-27T08:35:20.241434+0000","last_peered":"2024-05-27T08:35:20.241434+0000","last_clean":"2024-05-27T08:35:20.241434+0000","last_became_active":"2024-05-27T08:35:04.022458+0000","last_became_peered":"2024-05-27T08:35:04.022458+0000","last_unstale":"2024-05-27T08:35:20.241434+0000","last_undegraded":"2024-05-27T08:35:20.241434+0000","last_fullsized":"2024-05-27T08:35:20.241434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:01.029226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049736+0000","last_change":"2024-05-27T08:35:07.049136+0000","last_active":"2024-05-27T08:35:09.049736+0000","last_peered":"2024-05-27T08:35:09.049736+0000","last_clean":"2024-05-27T08:35:09.049736+0000","last_became_active":"2024-05-27T08:35:07.048793+0000","last_became_peered":"2024-05-27T08:35:07.048793+0000","last_unstale":"2024-05-27T08:35:09.049736+0000","last_undegraded":"2024-05-27T08:35:09.049736+0000","last_fullsized":"2024-05-27T08:35:09.049736+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:10:21.191743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:48.317805+0000","last_change":"2024-05-27T08:35:04.262102+0000","last_active":"2024-05-27T08:35:48.317805+0000","last_peered":"2024-05-27T08:35:48.317805+0000","last_clean":"2024-05-27T08:35:48.317805+0000","last_became_active":"2024-05-27T08:35:04.261665+0000","last_became_peered":"2024-05-27T08:35:04.261665+0000","last_unstale":"2024-05-27T08:35:48.317805+0000","last_undegraded":"2024-05-27T08:35:48.317805+0000","last_fullsized":"2024-05-27T08:35:48.317805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:57:57.712197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050329+0000","last_change":"2024-05-27T08:35:07.051557+0000","last_active":"2024-05-27T08:35:09.050329+0000","last_peered":"2024-05-27T08:35:09.050329+0000","last_clean":"2024-05-27T08:35:09.050329+0000","last_became_active":"2024-05-27T08:35:07.051246+0000","last_became_peered":"2024-05-27T08:35:07.051246+0000","last_unstale":"2024-05-27T08:35:09.050329+0000","last_undegraded":"2024-05-27T08:35:09.050329+0000","last_fullsized":"2024-05-27T08:35:09.050329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:08:13.248301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.273758+0000","last_change":"2024-05-27T08:35:04.022328+0000","last_active":"2024-05-27T08:35:23.273758+0000","last_peered":"2024-05-27T08:35:23.273758+0000","last_clean":"2024-05-27T08:35:23.273758+0000","last_became_active":"2024-05-27T08:35:04.021711+0000","last_became_peered":"2024-05-27T08:35:04.021711+0000","last_unstale":"2024-05-27T08:35:23.273758+0000","last_undegraded":"2024-05-27T08:35:23.273758+0000","last_fullsized":"2024-05-27T08:35:23.273758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:32.841474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.049729+0000","last_change":"2024-05-27T08:35:07.056768+0000","last_active":"2024-05-27T08:35:09.049729+0000","last_peered":"2024-05-27T08:35:09.049729+0000","last_clean":"2024-05-27T08:35:09.049729+0000","last_became_active":"2024-05-27T08:35:07.056482+0000","last_became_peered":"2024-05-27T08:35:07.056482+0000","last_unstale":"2024-05-27T08:35:09.049729+0000","last_undegraded":"2024-05-27T08:35:09.049729+0000","last_fullsized":"2024-05-27T08:35:09.049729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:23.515625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:23.271596+0000","last_change":"2024-05-27T08:35:04.016613+0000","last_active":"2024-05-27T08:35:23.271596+0000","last_peered":"2024-05-27T08:35:23.271596+0000","last_clean":"2024-05-27T08:35:23.271596+0000","last_became_active":"2024-05-27T08:35:04.016268+0000","last_became_peered":"2024-05-27T08:35:04.016268+0000","last_unstale":"2024-05-27T08:35:23.271596+0000","last_undegraded":"2024-05-27T08:35:23.271596+0000","last_fullsized":"2024-05-27T08:35:23.271596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:11.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050355+0000","last_change":"2024-05-27T08:35:07.051515+0000","last_active":"2024-05-27T08:35:09.050355+0000","last_peered":"2024-05-27T08:35:09.050355+0000","last_clean":"2024-05-27T08:35:09.050355+0000","last_became_active":"2024-05-27T08:35:07.051197+0000","last_became_peered":"2024-05-27T08:35:07.051197+0000","last_unstale":"2024-05-27T08:35:09.050355+0000","last_undegraded":"2024-05-27T08:35:09.050355+0000","last_fullsized":"2024-05-27T08:35:09.050355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:49.233474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050330+0000","last_change":"2024-05-27T08:35:07.051960+0000","last_active":"2024-05-27T08:35:09.050330+0000","last_peered":"2024-05-27T08:35:09.050330+0000","last_clean":"2024-05-27T08:35:09.050330+0000","last_became_active":"2024-05-27T08:35:07.051395+0000","last_became_peered":"2024-05-27T08:35:07.051395+0000","last_unstale":"2024-05-27T08:35:09.050330+0000","last_undegraded":"2024-05-27T08:35:09.050330+0000","last_fullsized":"2024-05-27T08:35:09.050330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:57.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042574+0000","last_change":"2024-05-27T08:35:07.050096+0000","last_active":"2024-05-27T08:35:09.042574+0000","last_peered":"2024-05-27T08:35:09.042574+0000","last_clean":"2024-05-27T08:35:09.042574+0000","last_became_active":"2024-05-27T08:35:07.049755+0000","last_became_peered":"2024-05-27T08:35:07.049755+0000","last_unstale":"2024-05-27T08:35:09.042574+0000","last_undegraded":"2024-05-27T08:35:09.042574+0000","last_fullsized":"2024-05-27T08:35:09.042574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:39.768128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042494+0000","last_change":"2024-05-27T08:35:07.048281+0000","last_active":"2024-05-27T08:35:09.042494+0000","last_peered":"2024-05-27T08:35:09.042494+0000","last_clean":"2024-05-27T08:35:09.042494+0000","last_became_active":"2024-05-27T08:35:07.047879+0000","last_became_peered":"2024-05-27T08:35:07.047879+0000","last_unstale":"2024-05-27T08:35:09.042494+0000","last_undegraded":"2024-05-27T08:35:09.042494+0000","last_fullsized":"2024-05-27T08:35:09.042494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:43.631671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054655+0000","last_change":"2024-05-27T08:35:07.050501+0000","last_active":"2024-05-27T08:35:09.054655+0000","last_peered":"2024-05-27T08:35:09.054655+0000","last_clean":"2024-05-27T08:35:09.054655+0000","last_became_active":"2024-05-27T08:35:07.050159+0000","last_became_peered":"2024-05-27T08:35:07.050159+0000","last_unstale":"2024-05-27T08:35:09.054655+0000","last_undegraded":"2024-05-27T08:35:09.054655+0000","last_fullsized":"2024-05-27T08:35:09.054655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:17:23.293019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052179+0000","last_change":"2024-05-27T08:35:07.047450+0000","last_active":"2024-05-27T08:35:09.052179+0000","last_peered":"2024-05-27T08:35:09.052179+0000","last_clean":"2024-05-27T08:35:09.052179+0000","last_became_active":"2024-05-27T08:35:07.047094+0000","last_became_peered":"2024-05-27T08:35:07.047094+0000","last_unstale":"2024-05-27T08:35:09.052179+0000","last_undegraded":"2024-05-27T08:35:09.052179+0000","last_fullsized":"2024-05-27T08:35:09.052179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:12:11.882117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050942+0000","last_change":"2024-05-27T08:35:07.052382+0000","last_active":"2024-05-27T08:35:09.050942+0000","last_peered":"2024-05-27T08:35:09.050942+0000","last_clean":"2024-05-27T08:35:09.050942+0000","last_became_active":"2024-05-27T08:35:07.052049+0000","last_became_peered":"2024-05-27T08:35:07.052049+0000","last_unstale":"2024-05-27T08:35:09.050942+0000","last_undegraded":"2024-05-27T08:35:09.050942+0000","last_fullsized":"2024-05-27T08:35:09.050942+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:08.912872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.042910+0000","last_change":"2024-05-27T08:35:07.043810+0000","last_active":"2024-05-27T08:35:09.042910+0000","last_peered":"2024-05-27T08:35:09.042910+0000","last_clean":"2024-05-27T08:35:09.042910+0000","last_became_active":"2024-05-27T08:35:07.043449+0000","last_became_peered":"2024-05-27T08:35:07.043449+0000","last_unstale":"2024-05-27T08:35:09.042910+0000","last_undegraded":"2024-05-27T08:35:09.042910+0000","last_fullsized":"2024-05-27T08:35:09.042910+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:18.089405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050822+0000","last_change":"2024-05-27T08:35:07.043676+0000","last_active":"2024-05-27T08:35:09.050822+0000","last_peered":"2024-05-27T08:35:09.050822+0000","last_clean":"2024-05-27T08:35:09.050822+0000","last_became_active":"2024-05-27T08:35:07.043365+0000","last_became_peered":"2024-05-27T08:35:07.043365+0000","last_unstale":"2024-05-27T08:35:09.050822+0000","last_undegraded":"2024-05-27T08:35:09.050822+0000","last_fullsized":"2024-05-27T08:35:09.050822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:13:32.374612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.058160+0000","last_change":"2024-05-27T08:35:07.049960+0000","last_active":"2024-05-27T08:35:09.058160+0000","last_peered":"2024-05-27T08:35:09.058160+0000","last_clean":"2024-05-27T08:35:09.058160+0000","last_became_active":"2024-05-27T08:35:07.049595+0000","last_became_peered":"2024-05-27T08:35:07.049595+0000","last_unstale":"2024-05-27T08:35:09.058160+0000","last_undegraded":"2024-05-27T08:35:09.058160+0000","last_fullsized":"2024-05-27T08:35:09.058160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:18.182658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050739+0000","last_change":"2024-05-27T08:35:07.043790+0000","last_active":"2024-05-27T08:35:09.050739+0000","last_peered":"2024-05-27T08:35:09.050739+0000","last_clean":"2024-05-27T08:35:09.050739+0000","last_became_active":"2024-05-27T08:35:07.043476+0000","last_became_peered":"2024-05-27T08:35:07.043476+0000","last_unstale":"2024-05-27T08:35:09.050739+0000","last_undegraded":"2024-05-27T08:35:09.050739+0000","last_fullsized":"2024-05-27T08:35:09.050739+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:01.626979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.054997+0000","last_change":"2024-05-27T08:35:07.044928+0000","last_active":"2024-05-27T08:35:09.054997+0000","last_peered":"2024-05-27T08:35:09.054997+0000","last_clean":"2024-05-27T08:35:09.054997+0000","last_became_active":"2024-05-27T08:35:07.044577+0000","last_became_peered":"2024-05-27T08:35:07.044577+0000","last_unstale":"2024-05-27T08:35:09.054997+0000","last_undegraded":"2024-05-27T08:35:09.054997+0000","last_fullsized":"2024-05-27T08:35:09.054997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:36.560525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.052101+0000","last_change":"2024-05-27T08:35:07.045067+0000","last_active":"2024-05-27T08:35:09.052101+0000","last_peered":"2024-05-27T08:35:09.052101+0000","last_clean":"2024-05-27T08:35:09.052101+0000","last_became_active":"2024-05-27T08:35:07.044759+0000","last_became_peered":"2024-05-27T08:35:07.044759+0000","last_unstale":"2024-05-27T08:35:09.052101+0000","last_undegraded":"2024-05-27T08:35:09.052101+0000","last_fullsized":"2024-05-27T08:35:09.052101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:08.440135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-27T08:35:09.050351+0000","last_change":"2024-05-27T08:35:07.048186+0000","last_active":"2024-05-27T08:35:09.050351+0000","last_peered":"2024-05-27T08:35:09.050351+0000","last_clean":"2024-05-27T08:35:09.050351+0000","last_became_active":"2024-05-27T08:35:07.047828+0000","last_became_peered":"2024-05-27T08:35:07.047828+0000","last_unstale":"2024-05-27T08:35:09.050351+0000","last_undegraded":"2024-05-27T08:35:09.050351+0000","last_fullsized":"2024-05-27T08:35:09.050351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:09.928234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":292,"num_read_kb":220,"num_write":246,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":195503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":901120,"data_stored":430954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1601536,"data_stored":1082136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939681,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":430884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":431262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332138,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1519616,"data_stored":1024171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495664,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":909312,"data_stored":430601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":485881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":431940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":491887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117055,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":430426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920825,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":430853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1023458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T08:35:51.899 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T08:35:51.900 INFO:tasks.cephadm:Setting up client nodes... 2024-05-27T08:35:51.900 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-27T08:35:52.142 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:52.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:52 smithi077 ceph-mon[32007]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-05-27T08:35:52.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:52 smithi077 ceph-mon[32007]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:35:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:52 smithi179 ceph-mon[36481]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-05-27T08:35:52.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:52 smithi179 ceph-mon[36481]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:35:52.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:52 smithi092 ceph-mon[36049]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-05-27T08:35:52.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:52 smithi092 ceph-mon[36049]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:35:53.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 -- 172.21.15.77:0/1260407655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c103810 msgr2=0x7f7a6c10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 --2- 172.21.15.77:0/1260407655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c103810 0x7f7a6c10a0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a640099c0 tx=0x7f7a64009cd0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 -- 172.21.15.77:0/1260407655 shutdown_connections 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 --2- 172.21.15.77:0/1260407655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c103810 0x7f7a6c10a0c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 --2- 172.21.15.77:0/1260407655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a6c101c70 0x7f7a6c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 --2- 172.21.15.77:0/1260407655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6c102e70 0x7f7a6c1032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 -- 172.21.15.77:0/1260407655 >> 172.21.15.77:0/1260407655 conn(0x7f7a6c0fd210 msgr2=0x7f7a6c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 -- 172.21.15.77:0/1260407655 shutdown_connections 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 -- 172.21.15.77:0/1260407655 wait complete. 2024-05-27T08:35:53.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.621+0000 7f7a715b6700 1 Processor -- start 2024-05-27T08:35:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a715b6700 1 -- start start 2024-05-27T08:35:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a715b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c101c70 0x7f7a6c0ff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a715b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6c102e70 0x7f7a6c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a715b6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a6c103810 0x7f7a6c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a715b6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10d780 con 0x7f7a6c101c70 2024-05-27T08:35:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a715b6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10d600 con 0x7f7a6c102e70 2024-05-27T08:35:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a715b6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c10d900 con 0x7f7a6c103810 2024-05-27T08:35:53.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c101c70 0x7f7a6c0ff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c101c70 0x7f7a6c0ff2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49970/0 (socket says 172.21.15.77:49970) 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 -- 172.21.15.77:0/4121625530 learned_addr learned my addr 172.21.15.77:0/4121625530 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 -- 172.21.15.77:0/4121625530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a6c103810 msgr2=0x7f7a6c1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6a7fc700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6c102e70 0x7f7a6c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a6c103810 0x7f7a6c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 -- 172.21.15.77:0/4121625530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6c102e70 msgr2=0x7f7a6c19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6c102e70 0x7f7a6c19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 -- 172.21.15.77:0/4121625530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c1a1290 con 0x7f7a6c101c70 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6a7fc700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6c102e70 0x7f7a6c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.622+0000 7f7a6affd700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c101c70 0x7f7a6c0ff2b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00b9c0 tx=0x7f7a5c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.623+0000 7f7a53fff700 1 -- 172.21.15.77:0/4121625530 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c019070 con 0x7f7a6c101c70 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.623+0000 7f7a53fff700 1 -- 172.21.15.77:0/4121625530 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5c014440 con 0x7f7a6c101c70 2024-05-27T08:35:53.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.623+0000 7f7a53fff700 1 -- 172.21.15.77:0/4121625530 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0125a0 con 0x7f7a6c101c70 2024-05-27T08:35:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.623+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c1a1580 con 0x7f7a6c101c70 2024-05-27T08:35:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.623+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1a1ac0 con 0x7f7a6c101c70 2024-05-27T08:35:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.624+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6c109360 con 0x7f7a6c101c70 2024-05-27T08:35:53.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.625+0000 7f7a53fff700 1 -- 172.21.15.77:0/4121625530 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7a5c01c070 con 0x7f7a6c101c70 2024-05-27T08:35:53.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.626+0000 7f7a53fff700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7a540b2a30 0x7f7a540b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:53.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.626+0000 7f7a6a7fc700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7a540b2a30 0x7f7a540b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:53.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.626+0000 7f7a53fff700 1 -- 172.21.15.77:0/4121625530 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f7a5c0e1a40 con 0x7f7a6c101c70 2024-05-27T08:35:53.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.626+0000 7f7a6a7fc700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7a540b2a30 0x7f7a540b4ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7a60005fd0 tx=0x7f7a60005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:53.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.629+0000 7f7a53fff700 1 -- 172.21.15.77:0/4121625530 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a5c0a9840 con 0x7f7a6c101c70 2024-05-27T08:35:53.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.914+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7a6c02e5e0 con 0x7f7a6c101c70 2024-05-27T08:35:53.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.916+0000 7f7a53fff700 1 -- 172.21.15.77:0/4121625530 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f7a5c0a9140 con 0x7f7a6c101c70 2024-05-27T08:35:53.917 INFO:teuthology.orchestra.run.smithi077.stdout:[client.0] 2024-05-27T08:35:53.917 INFO:teuthology.orchestra.run.smithi077.stdout: key = AQDpRVRmysKONhAAdZh0jxxGDIya7SQBPio3Qg== 2024-05-27T08:35:53.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7a540b2a30 msgr2=0x7f7a540b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7a540b2a30 0x7f7a540b4ef0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7a60005fd0 tx=0x7f7a60005dc0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c101c70 msgr2=0x7f7a6c0ff2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c101c70 0x7f7a6c0ff2b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00b9c0 tx=0x7f7a5c00bd80 comp rx=0 tx=0).stop 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 shutdown_connections 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7a540b2a30 0x7f7a540b4ef0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7a6c103810 0x7f7a6c1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a6c101c70 0x7f7a6c0ff2b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 --2- 172.21.15.77:0/4121625530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a6c102e70 0x7f7a6c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 >> 172.21.15.77:0/4121625530 conn(0x7f7a6c0fd210 msgr2=0x7f7a6c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 shutdown_connections 2024-05-27T08:35:53.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:53.918+0000 7f7a715b6700 1 -- 172.21.15.77:0/4121625530 wait complete. 2024-05-27T08:35:54.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:54 smithi077 ceph-mon[32007]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T08:35:54.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:54 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/4121625530' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T08:35:54.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:54 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/4121625530' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T08:35:54.541 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:35:54.542 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-27T08:35:54.542 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-27T08:35:54.579 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-27T08:35:54.579 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-27T08:35:54.579 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-05-27T08:35:54.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:54 smithi179 ceph-mon[36481]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T08:35:54.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:54 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/4121625530' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T08:35:54.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:54 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/4121625530' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T08:35:54.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:54 smithi092 ceph-mon[36049]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-05-27T08:35:54.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:54 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/4121625530' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-27T08:35:54.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:54 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/4121625530' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-27T08:35:54.854 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:56.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 -- 172.21.15.77:0/3158501239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 msgr2=0x7f409c06bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:56.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/3158501239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 0x7f409c06bf30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4084009990 tx=0x7f4084009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 -- 172.21.15.77:0/3158501239 shutdown_connections 2024-05-27T08:35:56.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/3158501239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409c06c470 0x7f409c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/3158501239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f409c1095a0 0x7f409c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/3158501239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 0x7f409c06bf30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 -- 172.21.15.77:0/3158501239 >> 172.21.15.77:0/3158501239 conn(0x7f409c0793b0 msgr2=0x7f409c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:56.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 -- 172.21.15.77:0/3158501239 shutdown_connections 2024-05-27T08:35:56.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.340+0000 7f40a0b0a700 1 -- 172.21.15.77:0/3158501239 wait complete. 2024-05-27T08:35:56.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.341+0000 7f40a0b0a700 1 Processor -- start 2024-05-27T08:35:56.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.341+0000 7f40a0b0a700 1 -- start start 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.341+0000 7f40a0b0a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 0x7f409c102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.341+0000 7f40a0b0a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409c06c470 0x7f409c102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.341+0000 7f40a0b0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f409c1095a0 0x7f409c102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.341+0000 7f40a0b0a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1136d0 con 0x7f409c06bab0 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.341+0000 7f40a0b0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f409c113550 con 0x7f409c1095a0 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.341+0000 7f40a0b0a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f409c113850 con 0x7f409c06c470 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 0x7f409c102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 0x7f409c102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49996/0 (socket says 172.21.15.77:49996) 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 -- 172.21.15.77:0/1861099372 learned_addr learned my addr 172.21.15.77:0/1861099372 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f4099d9b700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409c06c470 0x7f409c102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409ad9d700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f409c1095a0 0x7f409c102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:56.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 -- 172.21.15.77:0/1861099372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409c06c470 msgr2=0x7f409c102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409c06c470 0x7f409c102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 -- 172.21.15.77:0/1861099372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f409c1095a0 msgr2=0x7f409c102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f409c1095a0 0x7f409c102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 -- 172.21.15.77:0/1861099372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c1a8150 con 0x7f409c06bab0 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f409a59c700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 0x7f409c102500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f40940060b0 tx=0x7f409400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f40937fe700 1 -- 172.21.15.77:0/1861099372 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409401a070 con 0x7f409c06bab0 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c1a8440 con 0x7f409c06bab0 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f40937fe700 1 -- 172.21.15.77:0/1861099372 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409400ddd0 con 0x7f409c06bab0 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f40937fe700 1 -- 172.21.15.77:0/1861099372 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094014720 con 0x7f409c06bab0 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.342+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a89a0 con 0x7f409c06bab0 2024-05-27T08:35:56.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.343+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c06a230 con 0x7f409c06bab0 2024-05-27T08:35:56.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.344+0000 7f40937fe700 1 -- 172.21.15.77:0/1861099372 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f409401d070 con 0x7f409c06bab0 2024-05-27T08:35:56.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.345+0000 7f40937fe700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f40880b2910 0x7f40880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:56.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.345+0000 7f4099d9b700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f40880b2910 0x7f40880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:56.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.345+0000 7f40937fe700 1 -- 172.21.15.77:0/1861099372 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7979+0+0 (secure 0 0 0) 0x7f40940e1a30 con 0x7f409c06bab0 2024-05-27T08:35:56.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.346+0000 7f4099d9b700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f40880b2910 0x7f40880b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f408400b5c0 tx=0x7f4084005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:56.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.350+0000 7f40937fe700 1 -- 172.21.15.77:0/1861099372 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4094010070 con 0x7f409c06bab0 2024-05-27T08:35:56.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:56 smithi179 ceph-mon[36481]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-27T08:35:56.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.617+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f409c02e560 con 0x7f409c06bab0 2024-05-27T08:35:56.620 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:56 smithi077 ceph-mon[32007]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-27T08:35:56.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.623+0000 7f40937fe700 1 -- 172.21.15.77:0/1861099372 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288647 (secure 0 0 0) 0x7f40940a97a0 con 0x7f409c06bab0 2024-05-27T08:35:56.629 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:35:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f40880b2910 msgr2=0x7f40880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f40880b2910 0x7f40880b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f408400b5c0 tx=0x7f4084005960 comp rx=0 tx=0).stop 2024-05-27T08:35:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 msgr2=0x7f409c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 0x7f409c102500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f40940060b0 tx=0x7f409400d3b0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 shutdown_connections 2024-05-27T08:35:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f409c1095a0 0x7f409c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f409c06bab0 0x7f409c102500 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f40880b2910 0x7f40880b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 --2- 172.21.15.77:0/1861099372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f409c06c470 0x7f409c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 >> 172.21.15.77:0/1861099372 conn(0x7f409c0793b0 msgr2=0x7f409c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 shutdown_connections 2024-05-27T08:35:56.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:56.626+0000 7f40a0b0a700 1 -- 172.21.15.77:0/1861099372 wait complete. 2024-05-27T08:35:56.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:56 smithi092 ceph-mon[36049]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-05-27T08:35:57.273 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":2118880698},{"type":"v1","addr":"172.21.15.77:6801","nonce":2118880698}]},"active_addr":"172.21.15.77:6801/2118880698","active_change":"2024-05-27T08:27:57.064912+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.77:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":3561834991}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":2410459695}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":4211521509}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":2484667947}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.77:0","nonce":2818070116}]}]} 2024-05-27T08:35:57.280 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-27T08:35:57.280 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-27T08:35:57.281 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:35:57.529 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1861099372' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-27T08:35:57.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-27T08:35:57.631 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:57 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-27T08:35:57.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1861099372' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T08:35:57.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-27T08:35:57.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-27T08:35:57.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-27T08:35:57.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-27T08:35:57.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-27T08:35:57.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T08:35:57.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-27T08:35:57.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-27T08:35:57.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:57 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-27T08:35:57.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1861099372' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-05-27T08:35:57.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:57 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]: dispatch 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-27T08:35:58.374 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-27T08:35:58.375 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:58 smithi077 ceph-mon[32007]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T08:35:58.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-27T08:35:58.756 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-27T08:35:58.756 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-27T08:35:58.756 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-27T08:35:58.756 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-27T08:35:58.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-27T08:35:58.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T08:35:58.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-27T08:35:58.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-27T08:35:58.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-27T08:35:58.757 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:58 smithi092 ceph-mon[36049]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T08:35:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 682 B/s wr, 9 op/s 2024-05-27T08:35:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-27T08:35:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-27T08:35:58.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-27T08:35:58.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2024-05-27T08:35:58.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-05-27T08:35:58.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-27T08:35:58.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-05-27T08:35:58.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-05-27T08:35:58.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 6]}]': finished 2024-05-27T08:35:58.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:58 smithi179 ceph-mon[36481]: osdmap e84: 12 total, 12 up, 12 in 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 -- 172.21.15.77:0/1231758101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 msgr2=0x7f04d01028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 --2- 172.21.15.77:0/1231758101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 0x7f04d01028b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009990 tx=0x7f04c4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 -- 172.21.15.77:0/1231758101 shutdown_connections 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 --2- 172.21.15.77:0/1231758101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04d0102df0 0x7f04d010f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 --2- 172.21.15.77:0/1231758101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d0108510 0x7f04d01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 --2- 172.21.15.77:0/1231758101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 0x7f04d01028b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 -- 172.21.15.77:0/1231758101 >> 172.21.15.77:0/1231758101 conn(0x7f04d00793b0 msgr2=0x7f04d00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 -- 172.21.15.77:0/1231758101 shutdown_connections 2024-05-27T08:35:59.195 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.192+0000 7f04d6212700 1 -- 172.21.15.77:0/1231758101 wait complete. 2024-05-27T08:35:59.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.193+0000 7f04d6212700 1 Processor -- start 2024-05-27T08:35:59.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.193+0000 7f04d6212700 1 -- start start 2024-05-27T08:35:59.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.193+0000 7f04d6212700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 0x7f04d019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:59.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.193+0000 7f04d6212700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04d0102df0 0x7f04d019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:59.197 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.193+0000 7f04d6212700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d0108510 0x7f04d0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.193+0000 7f04d6212700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01125e0 con 0x7f04d0102430 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.193+0000 7f04d6212700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0112460 con 0x7f04d0108510 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.193+0000 7f04d6212700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0112760 con 0x7f04d0102df0 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 0x7f04d019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 0x7f04d019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50008/0 (socket says 172.21.15.77:50008) 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 -- 172.21.15.77:0/3185169279 learned_addr learned my addr 172.21.15.77:0/3185169279 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04ceffd700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04d0102df0 0x7f04d019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cffff700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d0108510 0x7f04d0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 -- 172.21.15.77:0/3185169279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04d0102df0 msgr2=0x7f04d019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04d0102df0 0x7f04d019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 -- 172.21.15.77:0/3185169279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d0108510 msgr2=0x7f04d0199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:59.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d0108510 0x7f04d0199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 -- 172.21.15.77:0/3185169279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d019a170 con 0x7f04d0102430 2024-05-27T08:35:59.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04cf7fe700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 0x7f04d019f850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04c000d950 tx=0x7f04c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:59.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.194+0000 7f04ccff9700 1 -- 172.21.15.77:0/3185169279 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0016070 con 0x7f04d0102430 2024-05-27T08:35:59.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.195+0000 7f04ccff9700 1 -- 172.21.15.77:0/3185169279 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04c000fd40 con 0x7f04d0102430 2024-05-27T08:35:59.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.195+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d019a460 con 0x7f04d0102430 2024-05-27T08:35:59.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.195+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d01a6720 con 0x7f04d0102430 2024-05-27T08:35:59.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.196+0000 7f04ccff9700 1 -- 172.21.15.77:0/3185169279 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c000e3b0 con 0x7f04d0102430 2024-05-27T08:35:59.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.196+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d0103b50 con 0x7f04d0102430 2024-05-27T08:35:59.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.198+0000 7f04ccff9700 1 -- 172.21.15.77:0/3185169279 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f04c0019070 con 0x7f04d0102430 2024-05-27T08:35:59.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.199+0000 7f04ccff9700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f04b80b2860 0x7f04b80b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:35:59.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.199+0000 7f04ceffd700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f04b80b2860 0x7f04b80b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:35:59.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.199+0000 7f04ccff9700 1 -- 172.21.15.77:0/3185169279 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f04c00e1fb0 con 0x7f04d0102430 2024-05-27T08:35:59.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.200+0000 7f04ceffd700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f04b80b2860 0x7f04b80b4d20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04c400b5c0 tx=0x7f04c4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:35:59.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.204+0000 7f04ccff9700 1 -- 172.21.15.77:0/3185169279 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04c00a9c30 con 0x7f04d0102430 2024-05-27T08:35:59.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.422+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04d019af40 con 0x7f04d0102430 2024-05-27T08:35:59.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.424+0000 7f04ccff9700 1 -- 172.21.15.77:0/3185169279 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22676 (secure 0 0 0) 0x7f04c00c3e20 con 0x7f04d0102430 2024-05-27T08:35:59.426 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:35:59.427 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":85,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:35:58.361436+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f04b80b2860 msgr2=0x7f04b80b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f04b80b2860 0x7f04b80b4d20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f04c400b5c0 tx=0x7f04c4005960 comp rx=0 tx=0).stop 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 msgr2=0x7f04d019f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 0x7f04d019f850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04c000d950 tx=0x7f04c000dc60 comp rx=0 tx=0).stop 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 shutdown_connections 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04d0108510 0x7f04d0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04d0102430 0x7f04d019f850 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f04b80b2860 0x7f04b80b4d20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 --2- 172.21.15.77:0/3185169279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f04d0102df0 0x7f04d019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:35:59.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 >> 172.21.15.77:0/3185169279 conn(0x7f04d00793b0 msgr2=0x7f04d006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:35:59.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 shutdown_connections 2024-05-27T08:35:59.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:35:59.426+0000 7f04d6212700 1 -- 172.21.15.77:0/3185169279 wait complete. 2024-05-27T08:35:59.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:35:59 smithi077 ceph-mon[32007]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T08:35:59.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:35:59 smithi092 ceph-mon[36049]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T08:35:59.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:35:59 smithi179 ceph-mon[36481]: osdmap e85: 12 total, 12 up, 12 in 2024-05-27T08:35:59.963 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-27T08:35:59.963 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:36:00.206 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:00.588 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:00 smithi077 ceph-mon[32007]: pgmap v316: 97 pgs: 8 peering, 89 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-05-27T08:36:00.588 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:00 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3185169279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:36:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:00 smithi092 ceph-mon[36049]: pgmap v316: 97 pgs: 8 peering, 89 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-05-27T08:36:00.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:00 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3185169279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:36:00.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:00 smithi179 ceph-mon[36481]: pgmap v316: 97 pgs: 8 peering, 89 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s 2024-05-27T08:36:00.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:00 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3185169279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 -- 172.21.15.77:0/2150826582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806bcb0 msgr2=0x7f627806c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 --2- 172.21.15.77:0/2150826582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806bcb0 0x7f627806c130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6260009990 tx=0x7f6260009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 -- 172.21.15.77:0/2150826582 shutdown_connections 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 --2- 172.21.15.77:0/2150826582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f627806c670 0x7f6278110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 --2- 172.21.15.77:0/2150826582 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62781095e0 0x7f62781099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 --2- 172.21.15.77:0/2150826582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806bcb0 0x7f627806c130 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 -- 172.21.15.77:0/2150826582 >> 172.21.15.77:0/2150826582 conn(0x7f62780784b0 msgr2=0x7f62780788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 -- 172.21.15.77:0/2150826582 shutdown_connections 2024-05-27T08:36:01.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.885+0000 7f627d225700 1 -- 172.21.15.77:0/2150826582 wait complete. 2024-05-27T08:36:01.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627d225700 1 Processor -- start 2024-05-27T08:36:01.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627d225700 1 -- start start 2024-05-27T08:36:01.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627d225700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f627806bcb0 0x7f6278102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:01.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627d225700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806c670 0x7f6278102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:01.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627d225700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62781095e0 0x7f6278102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627d225700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f62781137c0 con 0x7f627806c670 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627d225700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f6278113640 con 0x7f62781095e0 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627d225700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6278113940 con 0x7f627806bcb0 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f626ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806c670 0x7f6278102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f626ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806c670 0x7f6278102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45758/0 (socket says 172.21.15.77:45758) 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f626ffff700 1 -- 172.21.15.77:0/308533750 learned_addr learned my addr 172.21.15.77:0/308533750 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f6276d9d700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f627806bcb0 0x7f6278102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.886+0000 7f627759e700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62781095e0 0x7f6278102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f626ffff700 1 -- 172.21.15.77:0/308533750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f627806bcb0 msgr2=0x7f6278102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f626ffff700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f627806bcb0 0x7f6278102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f626ffff700 1 -- 172.21.15.77:0/308533750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62781095e0 msgr2=0x7f6278102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f626ffff700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62781095e0 0x7f6278102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f626ffff700 1 -- 172.21.15.77:0/308533750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62781a8300 con 0x7f627806c670 2024-05-27T08:36:01.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f626ffff700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806c670 0x7f6278102a80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6260000c00 tx=0x7f62600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:01.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f6274d99700 1 -- 172.21.15.77:0/308533750 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6260027070 con 0x7f627806c670 2024-05-27T08:36:01.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f6274d99700 1 -- 172.21.15.77:0/308533750 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62600040e0 con 0x7f627806c670 2024-05-27T08:36:01.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62781a8590 con 0x7f627806c670 2024-05-27T08:36:01.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f6274d99700 1 -- 172.21.15.77:0/308533750 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626000f460 con 0x7f627806c670 2024-05-27T08:36:01.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.887+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62781a8a40 con 0x7f627806c670 2024-05-27T08:36:01.892 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.888+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f627810bae0 con 0x7f627806c670 2024-05-27T08:36:01.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.889+0000 7f6274d99700 1 -- 172.21.15.77:0/308533750 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f626002a070 con 0x7f627806c670 2024-05-27T08:36:01.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.890+0000 7f6274d99700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f62640b2990 0x7f62640b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:01.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.890+0000 7f6276d9d700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f62640b2990 0x7f62640b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:01.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.890+0000 7f6274d99700 1 -- 172.21.15.77:0/308533750 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f62600e3730 con 0x7f627806c670 2024-05-27T08:36:01.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.890+0000 7f6276d9d700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f62640b2990 0x7f62640b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6268009a10 tx=0x7f6268009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:01.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:01.893+0000 7f6274d99700 1 -- 172.21.15.77:0/308533750 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6260031070 con 0x7f627806c670 2024-05-27T08:36:02.105 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.102+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6278103a70 con 0x7f627806c670 2024-05-27T08:36:02.106 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.104+0000 7f6274d99700 1 -- 172.21.15.77:0/308533750 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22676 (secure 0 0 0) 0x7f6260013070 con 0x7f627806c670 2024-05-27T08:36:02.107 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:36:02.107 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":85,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:35:58.361436+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f62640b2990 msgr2=0x7f62640b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f62640b2990 0x7f62640b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6268009a10 tx=0x7f6268009190 comp rx=0 tx=0).stop 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806c670 msgr2=0x7f6278102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806c670 0x7f6278102a80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6260000c00 tx=0x7f62600047b0 comp rx=0 tx=0).stop 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 shutdown_connections 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f62781095e0 0x7f6278102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f62640b2990 0x7f62640b4e50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f627806bcb0 0x7f6278102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:02.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 --2- 172.21.15.77:0/308533750 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f627806c670 0x7f6278102a80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 >> 172.21.15.77:0/308533750 conn(0x7f62780784b0 msgr2=0x7f6278101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 shutdown_connections 2024-05-27T08:36:02.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:02.107+0000 7f627d225700 1 -- 172.21.15.77:0/308533750 wait complete. 2024-05-27T08:36:02.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:02 smithi077 ceph-mon[32007]: pgmap v317: 97 pgs: 8 peering, 89 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-27T08:36:02.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:02 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/308533750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:36:02.703 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-27T08:36:02.703 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-27T08:36:02.703 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-27T08:36:02.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-27T08:36:02.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-27T08:36:02.705 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-27T08:36:02.705 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-27T08:36:02.705 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-27T08:36:02.706 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-27T08:36:02.706 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-27T08:36:02.707 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-27T08:36:02.707 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-27T08:36:02.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:02 smithi092 ceph-mon[36049]: pgmap v317: 97 pgs: 8 peering, 89 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-27T08:36:02.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:02 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/308533750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:36:02.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:02 smithi179 ceph-mon[36481]: pgmap v317: 97 pgs: 8 peering, 89 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-27T08:36:02.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:02 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/308533750' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:36:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.414 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.423 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.423 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.426 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.434 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.655 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:03 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:36:03.729 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:03 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:36:03.732 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:03 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:36:03.748 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.748 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.749 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.756 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:03.756 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:04.622 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:04 smithi077 ceph-mon[32007]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 170 B/s, 2 objects/s recovering 2024-05-27T08:36:04.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:04 smithi092 ceph-mon[36049]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 170 B/s, 2 objects/s recovering 2024-05-27T08:36:04.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:04 smithi179 ceph-mon[36481]: pgmap v318: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 170 B/s, 2 objects/s recovering 2024-05-27T08:36:05.573 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.573 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.573 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.574 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:05 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:05 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:05.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:05 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:06.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:06 smithi077 ceph-mon[32007]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 170 B/s, 2 objects/s recovering 2024-05-27T08:36:06.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:06 smithi179 ceph-mon[36481]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 170 B/s, 2 objects/s recovering 2024-05-27T08:36:06.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:06 smithi092 ceph-mon[36049]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 170 B/s, 2 objects/s recovering 2024-05-27T08:36:08.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:08 smithi077 ceph-mon[32007]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 139 B/s, 1 objects/s recovering 2024-05-27T08:36:08.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:08 smithi179 ceph-mon[36481]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 139 B/s, 1 objects/s recovering 2024-05-27T08:36:08.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:08 smithi092 ceph-mon[36049]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s; 139 B/s, 1 objects/s recovering 2024-05-27T08:36:10.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:10 smithi077 ceph-mon[32007]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 126 B/s, 1 objects/s recovering 2024-05-27T08:36:10.787 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:10 smithi092 ceph-mon[36049]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 126 B/s, 1 objects/s recovering 2024-05-27T08:36:10.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:10 smithi179 ceph-mon[36481]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 126 B/s, 1 objects/s recovering 2024-05-27T08:36:11.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 -- 172.21.15.77:0/195063049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89341095b0 msgr2=0x7f8934109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 --2- 172.21.15.77:0/195063049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89341095b0 0x7f8934109990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f892400b0a0 tx=0x7f892400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 -- 172.21.15.77:0/195063049 shutdown_connections 2024-05-27T08:36:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 --2- 172.21.15.77:0/195063049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893406bbe0 0x7f89341103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 --2- 172.21.15.77:0/195063049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89341095b0 0x7f8934109990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 --2- 172.21.15.77:0/195063049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f893406b220 0x7f893406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 -- 172.21.15.77:0/195063049 >> 172.21.15.77:0/195063049 conn(0x7f89340793f0 msgr2=0x7f8934079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 -- 172.21.15.77:0/195063049 shutdown_connections 2024-05-27T08:36:11.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.227+0000 7f8939094700 1 -- 172.21.15.77:0/195063049 wait complete. 2024-05-27T08:36:11.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8939094700 1 Processor -- start 2024-05-27T08:36:11.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8939094700 1 -- start start 2024-05-27T08:36:11.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8939094700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893406b220 0x7f893419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8939094700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f893406bbe0 0x7f893419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8939094700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89341095b0 0x7f8934199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8939094700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8934113760 con 0x7f893406bbe0 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8939094700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f89341135e0 con 0x7f89341095b0 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8939094700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89341138e0 con 0x7f893406b220 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f892a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f893406bbe0 0x7f893419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f892a59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f893406bbe0 0x7f893419fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36716/0 (socket says 172.21.15.77:36716) 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f892a59c700 1 -- 172.21.15.77:0/3123211363 learned_addr learned my addr 172.21.15.77:0/3123211363 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f893359e700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89341095b0 0x7f8934199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.228+0000 7f8932d9d700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893406b220 0x7f893419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f892a59c700 1 -- 172.21.15.77:0/3123211363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893406b220 msgr2=0x7f893419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f892a59c700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893406b220 0x7f893419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f892a59c700 1 -- 172.21.15.77:0/3123211363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89341095b0 msgr2=0x7f8934199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f892a59c700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89341095b0 0x7f8934199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f892a59c700 1 -- 172.21.15.77:0/3123211363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893419a200 con 0x7f893406bbe0 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f8932d9d700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f893406b220 0x7f893419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f893359e700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89341095b0 0x7f8934199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f892a59c700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f893406bbe0 0x7f893419fe20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b530 tx=0x7f891c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c015020 con 0x7f893406bbe0 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f891c011ce0 con 0x7f893406bbe0 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893419a4f0 con 0x7f893406bbe0 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c010d20 con 0x7f893406bbe0 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.229+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89341a67b0 con 0x7f893406bbe0 2024-05-27T08:36:11.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.230+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8934063cd0 con 0x7f893406bbe0 2024-05-27T08:36:11.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.231+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f891c018020 con 0x7f893406bbe0 2024-05-27T08:36:11.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.232+0000 7f8930d99700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f89200b2910 0x7f89200b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.232+0000 7f8932d9d700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f89200b2910 0x7f89200b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.232+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f891c0e21a0 con 0x7f893406bbe0 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.232+0000 7f8930d99700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] conn(0x7f89200bc4d0 0x7f89200be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.232+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 --> [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f891c00fec0 con 0x7f89200bc4d0 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.232+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f891c0e25d0 con 0x7f893406bbe0 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.232+0000 7f8932d9d700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f89200b2910 0x7f89200b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8924009e60 tx=0x7f892400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.232+0000 7f893359e700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] conn(0x7f89200bc4d0 0x7f89200be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.233+0000 7f893359e700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] conn(0x7f89200bc4d0 0x7f89200be970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:11.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.233+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 <== osd.0 v2:172.21.15.77:6802/3594583548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f891c00fec0 con 0x7f89200bc4d0 2024-05-27T08:36:11.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 --> [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8934064150 con 0x7f89200bc4d0 2024-05-27T08:36:11.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8930d99700 1 -- 172.21.15.77:0/3123211363 <== osd.0 v2:172.21.15.77:6802/3594583548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8934064150 con 0x7f89200bc4d0 2024-05-27T08:36:11.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] conn(0x7f89200bc4d0 msgr2=0x7f89200be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] conn(0x7f89200bc4d0 0x7f89200be970 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f89200b2910 msgr2=0x7f89200b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f89200b2910 0x7f89200b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8924009e60 tx=0x7f892400b7b0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f893406bbe0 msgr2=0x7f893419fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 --2- 172.21.15.77:0/3123211363 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f893406bbe0 0x7f893419fe20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b530 tx=0x7f891c00b8f0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f893359e700 1 -- 172.21.15.77:0/3123211363 reap_dead start 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 shutdown_connections 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.250+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 >> 172.21.15.77:0/3123211363 conn(0x7f89340793f0 msgr2=0x7f8934101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.251+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 shutdown_connections 2024-05-27T08:36:11.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.251+0000 7f8939094700 1 -- 172.21.15.77:0/3123211363 wait complete. 2024-05-27T08:36:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.478+0000 7efd9efbd700 1 -- 172.21.15.77:0/1982089357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 msgr2=0x7efd981028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.478+0000 7efd9efbd700 1 --2- 172.21.15.77:0/1982089357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 0x7efd981028b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd84009990 tx=0x7efd84009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.478+0000 7efd9efbd700 1 -- 172.21.15.77:0/1982089357 shutdown_connections 2024-05-27T08:36:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.478+0000 7efd9efbd700 1 --2- 172.21.15.77:0/1982089357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98102df0 0x7efd9810f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.478+0000 7efd9efbd700 1 --2- 172.21.15.77:0/1982089357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd98108510 0x7efd981088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.478+0000 7efd9efbd700 1 --2- 172.21.15.77:0/1982089357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 0x7efd981028b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.478+0000 7efd9efbd700 1 -- 172.21.15.77:0/1982089357 >> 172.21.15.77:0/1982089357 conn(0x7efd980793b0 msgr2=0x7efd980797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:11.493 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 -- 172.21.15.77:0/1982089357 shutdown_connections 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 -- 172.21.15.77:0/1982089357 wait complete. 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 Processor -- start 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 -- start start 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 0x7efd98198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd98102df0 0x7efd98198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98108510 0x7efd981992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd98112640 con 0x7efd98102430 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efd981124c0 con 0x7efd98102df0 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9efbd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efd981127c0 con 0x7efd98108510 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9cd59700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 0x7efd98198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9cd59700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 0x7efd98198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36732/0 (socket says 172.21.15.77:36732) 2024-05-27T08:36:11.494 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.479+0000 7efd9cd59700 1 -- 172.21.15.77:0/1415788771 learned_addr learned my addr 172.21.15.77:0/1415788771 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9d55a700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98108510 0x7efd981992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9cd59700 1 -- 172.21.15.77:0/1415788771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98108510 msgr2=0x7efd981992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9cd59700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98108510 0x7efd981992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9cd59700 1 -- 172.21.15.77:0/1415788771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd98102df0 msgr2=0x7efd98198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9cd59700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd98102df0 0x7efd98198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9cd59700 1 -- 172.21.15.77:0/1415788771 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd981a8090 con 0x7efd98102430 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9cd59700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 0x7efd98198850 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00d950 tx=0x7efd8c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c016070 con 0x7efd98102430 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd981a8380 con 0x7efd98102430 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd8c010460 con 0x7efd98102430 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00f620 con 0x7efd98102430 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.480+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd981a88c0 con 0x7efd98102430 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.481+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd98063cd0 con 0x7efd98102430 2024-05-27T08:36:11.495 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.482+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7efd8c019070 con 0x7efd98102430 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.483+0000 7efd95ffb700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7efd880b2910 0x7efd880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.483+0000 7efd97fff700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7efd880b2910 0x7efd880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.483+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7efd8c0e2560 con 0x7efd98102430 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.483+0000 7efd95ffb700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] conn(0x7efd880bc4d0 0x7efd880be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.483+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 --> [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd880bf040 con 0x7efd880bc4d0 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.483+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7efd8c0e2990 con 0x7efd98102430 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.483+0000 7efd97fff700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7efd880b2910 0x7efd880b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd840098e0 tx=0x7efd84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.483+0000 7efd9d55a700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] conn(0x7efd880bc4d0 0x7efd880be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.484+0000 7efd9d55a700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] conn(0x7efd880bc4d0 0x7efd880be970 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:11.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.485+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 <== osd.10 v2:172.21.15.92:6824/2191388192 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd880bf040 con 0x7efd880bc4d0 2024-05-27T08:36:11.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.501+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 --> [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd98064150 con 0x7efd880bc4d0 2024-05-27T08:36:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd95ffb700 1 -- 172.21.15.77:0/1415788771 <== osd.10 v2:172.21.15.92:6824/2191388192 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efd98064150 con 0x7efd880bc4d0 2024-05-27T08:36:11.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 >> [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] conn(0x7efd880bc4d0 msgr2=0x7efd880be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] conn(0x7efd880bc4d0 0x7efd880be970 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7efd880b2910 msgr2=0x7efd880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7efd880b2910 0x7efd880b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd840098e0 tx=0x7efd84005fb0 comp rx=0 tx=0).stop 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 msgr2=0x7efd98198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 --2- 172.21.15.77:0/1415788771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd98102430 0x7efd98198850 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00d950 tx=0x7efd8c00dd10 comp rx=0 tx=0).stop 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9d55a700 1 -- 172.21.15.77:0/1415788771 reap_dead start 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 shutdown_connections 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 >> 172.21.15.77:0/1415788771 conn(0x7efd980793b0 msgr2=0x7efd98100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 shutdown_connections 2024-05-27T08:36:11.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:11.502+0000 7efd9efbd700 1 -- 172.21.15.77:0/1415788771 wait complete. 2024-05-27T08:36:12.149 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:12.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 -- 172.21.15.77:0/3534042691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406bd10 msgr2=0x7f475406c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:12.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 --2- 172.21.15.77:0/3534042691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406bd10 0x7f475406c190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f47500099c0 tx=0x7f4750009cd0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 -- 172.21.15.77:0/3534042691 shutdown_connections 2024-05-27T08:36:12.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 --2- 172.21.15.77:0/3534042691 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f475406c6d0 0x7f4754110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 --2- 172.21.15.77:0/3534042691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4754109630 0x7f4754109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 --2- 172.21.15.77:0/3534042691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406bd10 0x7f475406c190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 -- 172.21.15.77:0/3534042691 >> 172.21.15.77:0/3534042691 conn(0x7f47540794e0 msgr2=0x7f47540798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:12.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 -- 172.21.15.77:0/3534042691 shutdown_connections 2024-05-27T08:36:12.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.343+0000 7f475c30d700 1 -- 172.21.15.77:0/3534042691 wait complete. 2024-05-27T08:36:12.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475c30d700 1 Processor -- start 2024-05-27T08:36:12.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475c30d700 1 -- start start 2024-05-27T08:36:12.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475c30d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f475406bd10 0x7f475419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:12.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475c30d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406c6d0 0x7f475419fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:12.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475c30d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754109630 0x7f4754199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:12.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475c30d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4754113790 con 0x7f475406c6d0 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475c30d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4754113610 con 0x7f4754109630 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475c30d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4754113910 con 0x7f475406bd10 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f47598a8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406c6d0 0x7f475419fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f47598a8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406c6d0 0x7f475419fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36744/0 (socket says 172.21.15.77:36744) 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f47598a8700 1 -- 172.21.15.77:0/835864699 learned_addr learned my addr 172.21.15.77:0/835864699 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475a8aa700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754109630 0x7f4754199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f475a0a9700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f475406bd10 0x7f475419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.344+0000 7f47598a8700 1 -- 172.21.15.77:0/835864699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f475406bd10 msgr2=0x7f475419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f47598a8700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f475406bd10 0x7f475419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f47598a8700 1 -- 172.21.15.77:0/835864699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754109630 msgr2=0x7f4754199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f47598a8700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4754109630 0x7f4754199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f47598a8700 1 -- 172.21.15.77:0/835864699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475419a250 con 0x7f475406c6d0 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f475a0a9700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f475406bd10 0x7f475419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f47598a8700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406c6d0 0x7f475419fe20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f475000b580 tx=0x7f47500037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:12.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750027070 con 0x7f475406c6d0 2024-05-27T08:36:12.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475419a4e0 con 0x7f475406c6d0 2024-05-27T08:36:12.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4750004110 con 0x7f475406c6d0 2024-05-27T08:36:12.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47541a67b0 con 0x7f475406c6d0 2024-05-27T08:36:12.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.345+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475000f460 con 0x7f475406c6d0 2024-05-27T08:36:12.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.346+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4754063cd0 con 0x7f475406c6d0 2024-05-27T08:36:12.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.348+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f475002a070 con 0x7f475406c6d0 2024-05-27T08:36:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.349+0000 7f474b7fe700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f47400b2910 0x7f47400b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.349+0000 7f475a0a9700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f47400b2910 0x7f47400b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:12.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.349+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f47500e3a90 con 0x7f475406c6d0 2024-05-27T08:36:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.350+0000 7f474b7fe700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] conn(0x7f47400bc4d0 0x7f47400be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.350+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 --> [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47500043d0 con 0x7f47400bc4d0 2024-05-27T08:36:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.350+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f47500e3ec0 con 0x7f475406c6d0 2024-05-27T08:36:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.350+0000 7f475a0a9700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f47400b2910 0x7f47400b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4744005fd0 tx=0x7f4744005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.350+0000 7f475a8aa700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] conn(0x7f47400bc4d0 0x7f47400be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:12.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.350+0000 7f475a8aa700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] conn(0x7f47400bc4d0 0x7f47400be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:12.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.351+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 <== osd.1 v2:172.21.15.92:6800/4268921047 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47500043d0 con 0x7f47400bc4d0 2024-05-27T08:36:12.371 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.370+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 --> [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f47540640d0 con 0x7f47400bc4d0 2024-05-27T08:36:12.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.371+0000 7f474b7fe700 1 -- 172.21.15.77:0/835864699 <== osd.1 v2:172.21.15.92:6800/4268921047 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f47540640d0 con 0x7f47400bc4d0 2024-05-27T08:36:12.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.371+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 >> [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] conn(0x7f47400bc4d0 msgr2=0x7f47400be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:12.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.371+0000 7f475c30d700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] conn(0x7f47400bc4d0 0x7f47400be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.371+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f47400b2910 msgr2=0x7f47400b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:12.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.371+0000 7f475c30d700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f47400b2910 0x7f47400b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4744005fd0 tx=0x7f4744005de0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.371+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406c6d0 msgr2=0x7f475419fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:12.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.371+0000 7f475c30d700 1 --2- 172.21.15.77:0/835864699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f475406c6d0 0x7f475419fe20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f475000b580 tx=0x7f47500037c0 comp rx=0 tx=0).stop 2024-05-27T08:36:12.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.371+0000 7f475a8aa700 1 -- 172.21.15.77:0/835864699 reap_dead start 2024-05-27T08:36:12.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.372+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 shutdown_connections 2024-05-27T08:36:12.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.372+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 >> 172.21.15.77:0/835864699 conn(0x7f47540794e0 msgr2=0x7f4754100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:12.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.372+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 shutdown_connections 2024-05-27T08:36:12.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:12.372+0000 7f475c30d700 1 -- 172.21.15.77:0/835864699 wait complete. 2024-05-27T08:36:12.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:12 smithi077 ceph-mon[32007]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 113 B/s, 1 objects/s recovering 2024-05-27T08:36:12.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:12 smithi179 ceph-mon[36481]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 113 B/s, 1 objects/s recovering 2024-05-27T08:36:12.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:12 smithi092 ceph-mon[36049]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 113 B/s, 1 objects/s recovering 2024-05-27T08:36:14.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:14 smithi077 ceph-mon[32007]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 113 B/s, 1 objects/s recovering 2024-05-27T08:36:14.772 INFO:teuthology.orchestra.run.smithi077.stdout:296352743458 2024-05-27T08:36:14.773 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-27T08:36:14.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:14 smithi179 ceph-mon[36481]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 113 B/s, 1 objects/s recovering 2024-05-27T08:36:14.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:14 smithi092 ceph-mon[36049]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s; 113 B/s, 1 objects/s recovering 2024-05-27T08:36:15.216 INFO:teuthology.orchestra.run.smithi077.stdout:34359738451 2024-05-27T08:36:15.217 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-27T08:36:15.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 -- 172.21.15.77:0/521785628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4102e00 msgr2=0x7f30a410f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 --2- 172.21.15.77:0/521785628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4102e00 0x7f30a410f2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f309c00b0a0 tx=0x7f309c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 -- 172.21.15.77:0/521785628 shutdown_connections 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 --2- 172.21.15.77:0/521785628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4102e00 0x7f30a410f2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 --2- 172.21.15.77:0/521785628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a4108520 0x7f30a4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 --2- 172.21.15.77:0/521785628 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a4102440 0x7f30a41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 -- 172.21.15.77:0/521785628 >> 172.21.15.77:0/521785628 conn(0x7f30a40793f0 msgr2=0x7f30a4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 -- 172.21.15.77:0/521785628 shutdown_connections 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.418+0000 7f30ab0ed700 1 -- 172.21.15.77:0/521785628 wait complete. 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.419+0000 7f30ab0ed700 1 Processor -- start 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.419+0000 7f30ab0ed700 1 -- start start 2024-05-27T08:36:15.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30ab0ed700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a4102440 0x7f30a419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30ab0ed700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a4102e00 0x7f30a419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30ab0ed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4108520 0x7f30a4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30ab0ed700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4112650 con 0x7f30a4108520 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30ab0ed700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41124d0 con 0x7f30a4102440 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30ab0ed700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41127d0 con 0x7f30a4102e00 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4108520 0x7f30a4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4108520 0x7f30a4199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36774/0 (socket says 172.21.15.77:36774) 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 -- 172.21.15.77:0/2111412886 learned_addr learned my addr 172.21.15.77:0/2111412886 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a3fff700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a4102e00 0x7f30a419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a8e89700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a4102440 0x7f30a419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 -- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a4102e00 msgr2=0x7f30a419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a4102e00 0x7f30a419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:15.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 -- 172.21.15.77:0/2111412886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a4102440 msgr2=0x7f30a419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a4102440 0x7f30a419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 -- 172.21.15.77:0/2111412886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a419a1c0 con 0x7f30a4108520 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a3fff700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30a4102e00 0x7f30a419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a8e89700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f30a4102440 0x7f30a419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a968a700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4108520 0x7f30a4199a60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f309c00b830 tx=0x7f309c004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c021070 con 0x7f30a4108520 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f309c003c00 con 0x7f30a4108520 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a419a450 con 0x7f30a4108520 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.420+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c004730 con 0x7f30a4108520 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.421+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41a6770 con 0x7f30a4108520 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.421+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f30a4063cd0 con 0x7f30a4108520 2024-05-27T08:36:15.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.423+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f309c003db0 con 0x7f30a4108520 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.424+0000 7f30a1ffb700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30940b2910 0x7f30940b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.424+0000 7f30a8e89700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30940b2910 0x7f30940b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.424+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f309c00f020 con 0x7f30a4108520 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.424+0000 7f30a1ffb700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] conn(0x7f30940bc4d0 0x7f30940be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.424+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 --> [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30940bf040 con 0x7f30940bc4d0 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.424+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f309c0e40e0 con 0x7f30a4108520 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.424+0000 7f30a8e89700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30940b2910 0x7f30940b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3098005950 tx=0x7f30980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.425+0000 7f30a3fff700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] conn(0x7f30940bc4d0 0x7f30940be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:15.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.425+0000 7f30a3fff700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] conn(0x7f30940bc4d0 0x7f30940be970 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.426+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 <== osd.11 v2:172.21.15.179:6824/113533236 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30940bf040 con 0x7f30940bc4d0 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.442+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 --> [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30a4064150 con 0x7f30940bc4d0 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.443+0000 7f30a1ffb700 1 -- 172.21.15.77:0/2111412886 <== osd.11 v2:172.21.15.179:6824/113533236 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f30a4064150 con 0x7f30940bc4d0 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] conn(0x7f30940bc4d0 msgr2=0x7f30940be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] conn(0x7f30940bc4d0 0x7f30940be970 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30940b2910 msgr2=0x7f30940b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f30940b2910 0x7f30940b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3098005950 tx=0x7f30980058e0 comp rx=0 tx=0).stop 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4108520 msgr2=0x7f30a4199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 --2- 172.21.15.77:0/2111412886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4108520 0x7f30a4199a60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f309c00b830 tx=0x7f309c004d90 comp rx=0 tx=0).stop 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30a968a700 1 -- 172.21.15.77:0/2111412886 reap_dead start 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 shutdown_connections 2024-05-27T08:36:15.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 >> 172.21.15.77:0/2111412886 conn(0x7f30a40793f0 msgr2=0x7f30a406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:15.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 shutdown_connections 2024-05-27T08:36:15.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:15.444+0000 7f30ab0ed700 1 -- 172.21.15.77:0/2111412886 wait complete. 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 -- 172.21.15.77:0/495864706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b0102e20 msgr2=0x7f23b010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 --2- 172.21.15.77:0/495864706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b0102e20 0x7f23b010f330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f23a800b0a0 tx=0x7f23a800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 -- 172.21.15.77:0/495864706 shutdown_connections 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 --2- 172.21.15.77:0/495864706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b0102e20 0x7f23b010f330 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 --2- 172.21.15.77:0/495864706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b01084f0 0x7f23b01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 --2- 172.21.15.77:0/495864706 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102460 0x7f23b01028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 -- 172.21.15.77:0/495864706 >> 172.21.15.77:0/495864706 conn(0x7f23b00794e0 msgr2=0x7f23b00798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 -- 172.21.15.77:0/495864706 shutdown_connections 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 -- 172.21.15.77:0/495864706 wait complete. 2024-05-27T08:36:16.450 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 Processor -- start 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.108+0000 7f23b6157700 1 -- start start 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23b6157700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102460 0x7f23b019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23b6157700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b0102e20 0x7f23b019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23b6157700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b01084f0 0x7f23b01a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23b6157700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f23b0112630 con 0x7f23b01084f0 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23b6157700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01124b0 con 0x7f23b0102e20 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23b6157700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23b01127b0 con 0x7f23b0102460 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b01084f0 0x7f23b01a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b01084f0 0x7f23b01a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36762/0 (socket says 172.21.15.77:36762) 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 -- 172.21.15.77:0/256422952 learned_addr learned my addr 172.21.15.77:0/256422952 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23aeffd700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b0102e20 0x7f23b019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23af7fe700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102460 0x7f23b019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:16.451 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 -- 172.21.15.77:0/256422952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102460 msgr2=0x7f23b019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23b0102460 0x7f23b019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 -- 172.21.15.77:0/256422952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b0102e20 msgr2=0x7f23b019b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23b0102e20 0x7f23b019b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 -- 172.21.15.77:0/256422952 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b01a60b0 con 0x7f23b01084f0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23affff700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b01084f0 0x7f23b01a5b70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f23a800b830 tx=0x7f23a8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8021070 con 0x7f23b01084f0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23a8007e30 con 0x7f23b01084f0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a8004700 con 0x7f23b01084f0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23b01a6280 con 0x7f23b01084f0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.109+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23b01a66d0 con 0x7f23b01084f0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.110+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23b0063c10 con 0x7f23b01084f0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.112+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f23a8003d70 con 0x7f23b01084f0 2024-05-27T08:36:16.452 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.112+0000 7f23acff9700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23980b2990 0x7f23980b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:16.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.112+0000 7f23af7fe700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23980b2990 0x7f23980b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:16.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.112+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f23a800f020 con 0x7f23b01084f0 2024-05-27T08:36:16.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.113+0000 7f23acff9700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] conn(0x7f23980bc550 0x7f23980be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:16.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.113+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 --> [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23980bf0c0 con 0x7f23980bc550 2024-05-27T08:36:16.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.113+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f23a80e3e40 con 0x7f23b01084f0 2024-05-27T08:36:16.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.113+0000 7f23af7fe700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23980b2990 0x7f23980b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23a0005950 tx=0x7f23a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:16.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.113+0000 7f23aeffd700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] conn(0x7f23980bc550 0x7f23980be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.113+0000 7f23aeffd700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] conn(0x7f23980bc550 0x7f23980be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.114+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 <== osd.7 v2:172.21.15.92:6816/3593066168 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23980bf0c0 con 0x7f23980bc550 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.131+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 --> [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23b0064010 con 0x7f23980bc550 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.131+0000 7f23acff9700 1 -- 172.21.15.77:0/256422952 <== osd.7 v2:172.21.15.92:6816/3593066168 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f23b0064010 con 0x7f23980bc550 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 >> [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] conn(0x7f23980bc550 msgr2=0x7f23980be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] conn(0x7f23980bc550 0x7f23980be9f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23980b2990 msgr2=0x7f23980b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23980b2990 0x7f23980b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23a0005950 tx=0x7f23a000b410 comp rx=0 tx=0).stop 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b01084f0 msgr2=0x7f23b01a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 --2- 172.21.15.77:0/256422952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f23b01084f0 0x7f23b01a5b70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f23a800b830 tx=0x7f23a8004ce0 comp rx=0 tx=0).stop 2024-05-27T08:36:16.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23affff700 1 -- 172.21.15.77:0/256422952 reap_dead start 2024-05-27T08:36:16.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 shutdown_connections 2024-05-27T08:36:16.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 >> 172.21.15.77:0/256422952 conn(0x7f23b00794e0 msgr2=0x7f23b01050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:16.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 shutdown_connections 2024-05-27T08:36:16.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:14.132+0000 7f23b6157700 1 -- 172.21.15.77:0/256422952 wait complete. 2024-05-27T08:36:16.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:16 smithi179 ceph-mon[36481]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:16.863 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:16 smithi077 ceph-mon[32007]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:16.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:16 smithi092 ceph-mon[36049]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:17.181 INFO:teuthology.orchestra.run.smithi077.stdout:55834574927 2024-05-27T08:36:17.182 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-27T08:36:17.953 INFO:teuthology.orchestra.run.smithi077.stdout:322122547229 2024-05-27T08:36:17.953 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-27T08:36:18.703 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:18 smithi077 ceph-mon[32007]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:18.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:18 smithi179 ceph-mon[36481]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:18.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:18 smithi092 ceph-mon[36049]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:20.641 INFO:teuthology.orchestra.run.smithi077.stdout:219043332144 2024-05-27T08:36:20.642 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-27T08:36:20.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.527+0000 7f9728bc0700 1 -- 172.21.15.77:0/4248019974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724102df0 msgr2=0x7f972410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:20.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.527+0000 7f9728bc0700 1 --2- 172.21.15.77:0/4248019974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724102df0 0x7f972410f2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f971c00b0a0 tx=0x7f971c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:20.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.527+0000 7f9728bc0700 1 -- 172.21.15.77:0/4248019974 shutdown_connections 2024-05-27T08:36:20.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.527+0000 7f9728bc0700 1 --2- 172.21.15.77:0/4248019974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724102df0 0x7f972410f2e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.527+0000 7f9728bc0700 1 --2- 172.21.15.77:0/4248019974 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724108510 0x7f97241088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.527+0000 7f9728bc0700 1 --2- 172.21.15.77:0/4248019974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9724102430 0x7f97241028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.527+0000 7f9728bc0700 1 -- 172.21.15.77:0/4248019974 >> 172.21.15.77:0/4248019974 conn(0x7f97240793b0 msgr2=0x7f97240797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.528+0000 7f9728bc0700 1 -- 172.21.15.77:0/4248019974 shutdown_connections 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.528+0000 7f9728bc0700 1 -- 172.21.15.77:0/4248019974 wait complete. 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.528+0000 7f9728bc0700 1 Processor -- start 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.528+0000 7f9728bc0700 1 -- start start 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9728bc0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724102430 0x7f972419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9728bc0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9724102df0 0x7f972419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9728bc0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724108510 0x7f97241977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9728bc0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9724111550 con 0x7f9724108510 2024-05-27T08:36:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9728bc0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f97241113d0 con 0x7f9724102430 2024-05-27T08:36:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9728bc0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97241116d0 con 0x7f9724102df0 2024-05-27T08:36:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724108510 0x7f97241977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724108510 0x7f97241977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36790/0 (socket says 172.21.15.77:36790) 2024-05-27T08:36:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 -- 172.21.15.77:0/3889365282 learned_addr learned my addr 172.21.15.77:0/3889365282 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9721d9b700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9724102df0 0x7f972419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f972259c700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724102430 0x7f972419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 -- 172.21.15.77:0/3889365282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9724102df0 msgr2=0x7f972419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9724102df0 0x7f972419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 -- 172.21.15.77:0/3889365282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724102430 msgr2=0x7f972419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724102430 0x7f972419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 -- 172.21.15.77:0/3889365282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9724197fd0 con 0x7f9724108510 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f972259c700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9724102430 0x7f972419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f9722d9d700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724108510 0x7f97241977f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f971c00b780 tx=0x7f971c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.529+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c021070 con 0x7f9724108510 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.530+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9724198260 con 0x7f9724108510 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.530+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971c007e30 con 0x7f9724108510 2024-05-27T08:36:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.530+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c0047b0 con 0x7f9724108510 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.530+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97241a4480 con 0x7f9724108510 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.530+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9724063cd0 con 0x7f9724108510 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.532+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f971c007980 con 0x7f9724108510 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f97137fe700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f970c0b2910 0x7f970c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f972259c700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f970c0b2910 0x7f970c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f971c0e36d0 con 0x7f9724108510 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f97137fe700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] conn(0x7f970c0bc4d0 0x7f970c0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 --> [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f970c0bf040 con 0x7f970c0bc4d0 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f971c0e3b00 con 0x7f9724108510 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f972259c700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f970c0b2910 0x7f970c0b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9714009c90 tx=0x7f9714009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f9721d9b700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] conn(0x7f970c0bc4d0 0x7f970c0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.533+0000 7f9721d9b700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] conn(0x7f970c0bc4d0 0x7f970c0be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.534+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 <== osd.3 v2:172.21.15.77:6810/2112674768 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f970c0bf040 con 0x7f970c0bc4d0 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.552+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 --> [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f97240640d0 con 0x7f970c0bc4d0 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.552+0000 7f97137fe700 1 -- 172.21.15.77:0/3889365282 <== osd.3 v2:172.21.15.77:6810/2112674768 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f97240640d0 con 0x7f970c0bc4d0 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.552+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] conn(0x7f970c0bc4d0 msgr2=0x7f970c0be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] conn(0x7f970c0bc4d0 0x7f970c0be970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f970c0b2910 msgr2=0x7f970c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f970c0b2910 0x7f970c0b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9714009c90 tx=0x7f9714009260 comp rx=0 tx=0).stop 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724108510 msgr2=0x7f97241977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 --2- 172.21.15.77:0/3889365282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9724108510 0x7f97241977f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f971c00b780 tx=0x7f971c004d00 comp rx=0 tx=0).stop 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9722d9d700 1 -- 172.21.15.77:0/3889365282 reap_dead start 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 shutdown_connections 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 >> 172.21.15.77:0/3889365282 conn(0x7f97240793b0 msgr2=0x7f9724100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 shutdown_connections 2024-05-27T08:36:20.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:18.553+0000 7f9728bc0700 1 -- 172.21.15.77:0/3889365282 wait complete. 2024-05-27T08:36:20.855 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:20 smithi077 ceph-mon[32007]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:20.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:20 smithi179 ceph-mon[36481]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:20.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:20 smithi092 ceph-mon[36049]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:21.700 INFO:teuthology.orchestra.run.smithi077.stdout:115964117061 2024-05-27T08:36:21.700 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-27T08:36:22.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 -- 172.21.15.77:0/211439990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3741035a0 msgr2=0x7fd37410f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 --2- 172.21.15.77:0/211439990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3741035a0 0x7fd37410f2b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00b0d0 tx=0x7fd36c00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 -- 172.21.15.77:0/211439990 shutdown_connections 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 --2- 172.21.15.77:0/211439990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3741035a0 0x7fd37410f2b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 --2- 172.21.15.77:0/211439990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3741074a0 0x7fd374107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 --2- 172.21.15.77:0/211439990 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd374102be0 0x7fd374103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 -- 172.21.15.77:0/211439990 >> 172.21.15.77:0/211439990 conn(0x7fd374078a40 msgr2=0x7fd3740ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 -- 172.21.15.77:0/211439990 shutdown_connections 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.103+0000 7fd378a52700 1 -- 172.21.15.77:0/211439990 wait complete. 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd378a52700 1 Processor -- start 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd378a52700 1 -- start start 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd378a52700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd374102be0 0x7fd37419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:22.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd378a52700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3741035a0 0x7fd37419b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd378a52700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3741074a0 0x7fd3741a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd378a52700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741126b0 con 0x7fd374102be0 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd378a52700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd374112530 con 0x7fd3741074a0 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd378a52700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd374112830 con 0x7fd3741035a0 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd37259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd374102be0 0x7fd37419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd37259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd374102be0 0x7fd37419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51840/0 (socket says 172.21.15.77:51840) 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd37259c700 1 -- 172.21.15.77:0/457008976 learned_addr learned my addr 172.21.15.77:0/457008976 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd372d9d700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3741074a0 0x7fd3741a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.104+0000 7fd36bfff700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3741035a0 0x7fd37419b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd37259c700 1 -- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3741035a0 msgr2=0x7fd37419b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd37259c700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3741035a0 0x7fd37419b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd37259c700 1 -- 172.21.15.77:0/457008976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3741074a0 msgr2=0x7fd3741a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:22.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd37259c700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3741074a0 0x7fd3741a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:22.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd37259c700 1 -- 172.21.15.77:0/457008976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3741a60f0 con 0x7fd374102be0 2024-05-27T08:36:22.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd372d9d700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd3741074a0 0x7fd3741a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:22.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd36bfff700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3741035a0 0x7fd37419b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:22.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd37259c700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd374102be0 0x7fd37419ab30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd36400b530 tx=0x7fd36400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:22.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364015020 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3741a62c0 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd364011ce0 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364010d20 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.105+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741a6800 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.106+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd374063c50 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.107+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd364018020 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.108+0000 7fd36b7fe700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd3600b2910 0x7fd3600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.108+0000 7fd36bfff700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd3600b2910 0x7fd3600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.108+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd3640e2280 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.108+0000 7fd36b7fe700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] conn(0x7fd3600bc4d0 0x7fd3600be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.108+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 --> [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd36401be50 con 0x7fd3600bc4d0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.108+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd3640e26b0 con 0x7fd374102be0 2024-05-27T08:36:22.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.108+0000 7fd36bfff700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd3600b2910 0x7fd3600b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd374101040 tx=0x7fd35c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.109+0000 7fd372d9d700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] conn(0x7fd3600bc4d0 0x7fd3600be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.109+0000 7fd372d9d700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] conn(0x7fd3600bc4d0 0x7fd3600be970 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.110+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 <== osd.2 v2:172.21.15.179:6800/2614728525 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd36401be50 con 0x7fd3600bc4d0 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.126+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 --> [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd3740640d0 con 0x7fd3600bc4d0 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd36b7fe700 1 -- 172.21.15.77:0/457008976 <== osd.2 v2:172.21.15.179:6800/2614728525 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd3740640d0 con 0x7fd3600bc4d0 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] conn(0x7fd3600bc4d0 msgr2=0x7fd3600be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd378a52700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] conn(0x7fd3600bc4d0 0x7fd3600be970 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd3600b2910 msgr2=0x7fd3600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd378a52700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd3600b2910 0x7fd3600b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd374101040 tx=0x7fd35c005f50 comp rx=0 tx=0).stop 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd374102be0 msgr2=0x7fd37419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd378a52700 1 --2- 172.21.15.77:0/457008976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd374102be0 0x7fd37419ab30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd36400b530 tx=0x7fd36400b8f0 comp rx=0 tx=0).stop 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd372d9d700 1 -- 172.21.15.77:0/457008976 reap_dead start 2024-05-27T08:36:22.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 shutdown_connections 2024-05-27T08:36:22.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.127+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 >> 172.21.15.77:0/457008976 conn(0x7fd374078a40 msgr2=0x7fd374105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:22.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.128+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 shutdown_connections 2024-05-27T08:36:22.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:22.128+0000 7fd378a52700 1 -- 172.21.15.77:0/457008976 wait complete. 2024-05-27T08:36:22.533 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:22 smithi077 ceph-mon[32007]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:22.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:22 smithi179 ceph-mon[36481]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:22.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:22 smithi092 ceph-mon[36049]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:23.073 INFO:teuthology.orchestra.run.smithi077.stdout:77309411403 2024-05-27T08:36:23.073 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-27T08:36:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.267+0000 7f5da83f5700 1 -- 172.21.15.77:0/1436061878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da0102e30 msgr2=0x7f5da010f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.267+0000 7f5da83f5700 1 --2- 172.21.15.77:0/1436061878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da0102e30 0x7f5da010f2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b0a0 tx=0x7f5d9800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 -- 172.21.15.77:0/1436061878 shutdown_connections 2024-05-27T08:36:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 --2- 172.21.15.77:0/1436061878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da0102e30 0x7f5da010f2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 --2- 172.21.15.77:0/1436061878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5da01084b0 0x7f5da0108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.269 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 --2- 172.21.15.77:0/1436061878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da0102470 0x7f5da01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 -- 172.21.15.77:0/1436061878 >> 172.21.15.77:0/1436061878 conn(0x7f5da00793f0 msgr2=0x7f5da0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:23.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 -- 172.21.15.77:0/1436061878 shutdown_connections 2024-05-27T08:36:23.270 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 -- 172.21.15.77:0/1436061878 wait complete. 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 Processor -- start 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.268+0000 7f5da83f5700 1 -- start start 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da83f5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5da0102470 0x7f5da019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da83f5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da0102e30 0x7f5da019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da83f5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da01084b0 0x7f5da0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da83f5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01125e0 con 0x7f5da01084b0 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da83f5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0112460 con 0x7f5da0102e30 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da83f5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0112760 con 0x7f5da0102470 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da01084b0 0x7f5da0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da01084b0 0x7f5da0198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51870/0 (socket says 172.21.15.77:51870) 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 -- 172.21.15.77:0/3188073843 learned_addr learned my addr 172.21.15.77:0/3188073843 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:23.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6191700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5da0102470 0x7f5da019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da5990700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da0102e30 0x7f5da019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 -- 172.21.15.77:0/3188073843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5da0102470 msgr2=0x7f5da019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5da0102470 0x7f5da019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 -- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da0102e30 msgr2=0x7f5da019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da0102e30 0x7f5da019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 -- 172.21.15.77:0/3188073843 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da0198fe0 con 0x7f5da01084b0 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da5990700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5da0102e30 0x7f5da019ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5da6992700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da01084b0 0x7f5da0198880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b830 tx=0x7f5d98004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98020070 con 0x7f5da01084b0 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.269+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d98007e30 con 0x7f5da01084b0 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.270+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da0199270 con 0x7f5da01084b0 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.270+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d980047d0 con 0x7f5da01084b0 2024-05-27T08:36:23.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.270+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da006bc70 con 0x7f5da01084b0 2024-05-27T08:36:23.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.270+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5da006c220 con 0x7f5da01084b0 2024-05-27T08:36:23.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.272+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5d98007980 con 0x7f5da01084b0 2024-05-27T08:36:23.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.273+0000 7f5d977fe700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5d8c0b2990 0x7f5d8c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.273+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5d9800f030 con 0x7f5da01084b0 2024-05-27T08:36:23.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.273+0000 7f5da6191700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5d8c0b2990 0x7f5d8c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.273+0000 7f5d977fe700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] conn(0x7f5d8c0bc550 0x7f5d8c0be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.273+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 --> [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d98015ec0 con 0x7f5d8c0bc550 2024-05-27T08:36:23.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.273+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f5d980e34e0 con 0x7f5da01084b0 2024-05-27T08:36:23.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.274+0000 7f5da6191700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5d8c0b2990 0x7f5d8c0b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5da01999c0 tx=0x7f5d9c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:23.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.274+0000 7f5da5990700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] conn(0x7f5d8c0bc550 0x7f5d8c0be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.274+0000 7f5da5990700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] conn(0x7f5d8c0bc550 0x7f5d8c0be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:23.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.275+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 <== osd.4 v2:172.21.15.92:6808/2588881840 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d98015ec0 con 0x7f5d8c0bc550 2024-05-27T08:36:23.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.291+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 --> [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5da006c620 con 0x7f5d8c0bc550 2024-05-27T08:36:23.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.292+0000 7f5d977fe700 1 -- 172.21.15.77:0/3188073843 <== osd.4 v2:172.21.15.92:6808/2588881840 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5da006c620 con 0x7f5d8c0bc550 2024-05-27T08:36:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.292+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] conn(0x7f5d8c0bc550 msgr2=0x7f5d8c0be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.292+0000 7f5da83f5700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] conn(0x7f5d8c0bc550 0x7f5d8c0be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5d8c0b2990 msgr2=0x7f5d8c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da83f5700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5d8c0b2990 0x7f5d8c0b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5da01999c0 tx=0x7f5d9c00b3c0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da01084b0 msgr2=0x7f5da0198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da83f5700 1 --2- 172.21.15.77:0/3188073843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5da01084b0 0x7f5da0198880 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b830 tx=0x7f5d98004ce0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da6992700 1 -- 172.21.15.77:0/3188073843 reap_dead start 2024-05-27T08:36:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 shutdown_connections 2024-05-27T08:36:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 >> 172.21.15.77:0/3188073843 conn(0x7f5da00793f0 msgr2=0x7f5da0100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 shutdown_connections 2024-05-27T08:36:23.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.293+0000 7f5da83f5700 1 -- 172.21.15.77:0/3188073843 wait complete. 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 -- 172.21.15.77:0/355896229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102470 msgr2=0x7f4ac01028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 --2- 172.21.15.77:0/355896229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102470 0x7f4ac01028f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009990 tx=0x7f4abc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 -- 172.21.15.77:0/355896229 shutdown_connections 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 --2- 172.21.15.77:0/355896229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ac0102e30 0x7f4ac010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 --2- 172.21.15.77:0/355896229 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac01084b0 0x7f4ac0108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 --2- 172.21.15.77:0/355896229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102470 0x7f4ac01028f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 -- 172.21.15.77:0/355896229 >> 172.21.15.77:0/355896229 conn(0x7f4ac0079410 msgr2=0x7f4ac0079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 -- 172.21.15.77:0/355896229 shutdown_connections 2024-05-27T08:36:23.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.982+0000 7f4ac80c0700 1 -- 172.21.15.77:0/355896229 wait complete. 2024-05-27T08:36:23.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac80c0700 1 Processor -- start 2024-05-27T08:36:23.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac80c0700 1 -- start start 2024-05-27T08:36:23.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac80c0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ac0102470 0x7f4ac0104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac80c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102e30 0x7f4ac01045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac80c0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac01084b0 0x7f4ac0104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac80c0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0112640 con 0x7f4ac0102e30 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac80c0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01124c0 con 0x7f4ac01084b0 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac80c0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01127c0 con 0x7f4ac0102470 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac565b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102e30 0x7f4ac01045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac565b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102e30 0x7f4ac01045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51884/0 (socket says 172.21.15.77:51884) 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac565b700 1 -- 172.21.15.77:0/2728016523 learned_addr learned my addr 172.21.15.77:0/2728016523 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac5e5c700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ac0102470 0x7f4ac0104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.983+0000 7f4ac665d700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac01084b0 0x7f4ac0104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac565b700 1 -- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ac0102470 msgr2=0x7f4ac0104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac565b700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ac0102470 0x7f4ac0104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac565b700 1 -- 172.21.15.77:0/2728016523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac01084b0 msgr2=0x7f4ac0104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:23.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac565b700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4ac01084b0 0x7f4ac0104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac565b700 1 -- 172.21.15.77:0/2728016523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac01a81f0 con 0x7f4ac0102e30 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac5e5c700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ac0102470 0x7f4ac0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac565b700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102e30 0x7f4ac01045c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4abc000c00 tx=0x7f4abc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc027070 con 0x7f4ac0102e30 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc003fa0 con 0x7f4ac0102e30 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc022db0 con 0x7f4ac0102e30 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac01a8480 con 0x7f4ac0102e30 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.984+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac01a8880 con 0x7f4ac0102e30 2024-05-27T08:36:23.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.985+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ac0063c50 con 0x7f4ac0102e30 2024-05-27T08:36:23.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.987+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4abc02a070 con 0x7f4ac0102e30 2024-05-27T08:36:23.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.987+0000 7f4ab6ffd700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4aac0b2910 0x7f4aac0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.988+0000 7f4ac5e5c700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4aac0b2910 0x7f4aac0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.988+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f4abc0e3760 con 0x7f4ac0102e30 2024-05-27T08:36:23.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.988+0000 7f4ab6ffd700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] conn(0x7f4aac0bc4d0 0x7f4aac0be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:23.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.988+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 --> [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4abc0043d0 con 0x7f4aac0bc4d0 2024-05-27T08:36:23.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.988+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f4abc0e3b10 con 0x7f4ac0102e30 2024-05-27T08:36:23.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.988+0000 7f4ac5e5c700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4aac0b2910 0x7f4aac0b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0005950 tx=0x7f4ab00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:23.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.988+0000 7f4ac665d700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] conn(0x7f4aac0bc4d0 0x7f4aac0be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:23.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.989+0000 7f4ac665d700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] conn(0x7f4aac0bc4d0 0x7f4aac0be970 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:23.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:23.989+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 <== osd.8 v2:172.21.15.179:6816/3383738414 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4abc0043d0 con 0x7f4aac0bc4d0 2024-05-27T08:36:24.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.007+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 --> [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ac00640d0 con 0x7f4aac0bc4d0 2024-05-27T08:36:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.007+0000 7f4ab6ffd700 1 -- 172.21.15.77:0/2728016523 <== osd.8 v2:172.21.15.179:6816/3383738414 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4ac00640d0 con 0x7f4aac0bc4d0 2024-05-27T08:36:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] conn(0x7f4aac0bc4d0 msgr2=0x7f4aac0be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:24.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] conn(0x7f4aac0bc4d0 0x7f4aac0be970 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4aac0b2910 msgr2=0x7f4aac0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f4aac0b2910 0x7f4aac0b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0005950 tx=0x7f4ab00058e0 comp rx=0 tx=0).stop 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102e30 msgr2=0x7f4ac01045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 --2- 172.21.15.77:0/2728016523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac0102e30 0x7f4ac01045c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4abc000c00 tx=0x7f4abc0047b0 comp rx=0 tx=0).stop 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac665d700 1 -- 172.21.15.77:0/2728016523 reap_dead start 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 shutdown_connections 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 >> 172.21.15.77:0/2728016523 conn(0x7f4ac0079410 msgr2=0x7f4ac006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 shutdown_connections 2024-05-27T08:36:24.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:24.008+0000 7f4ac80c0700 1 -- 172.21.15.77:0/2728016523 wait complete. 2024-05-27T08:36:24.436 INFO:teuthology.orchestra.run.smithi077.stdout:141733920832 2024-05-27T08:36:24.436 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-27T08:36:24.674 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:24 smithi077 ceph-mon[32007]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:24.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:24 smithi179 ceph-mon[36481]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:24.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:24 smithi092 ceph-mon[36049]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 -- 172.21.15.77:0/2759686324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581024d0 msgr2=0x7fd658102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 --2- 172.21.15.77:0/2759686324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581024d0 0x7fd658102950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009990 tx=0x7fd64c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:25.084 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 -- 172.21.15.77:0/2759686324 shutdown_connections 2024-05-27T08:36:25.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 --2- 172.21.15.77:0/2759686324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd658102e90 0x7fd65810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:25.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 --2- 172.21.15.77:0/2759686324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6581084d0 0x7fd6581088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:25.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 --2- 172.21.15.77:0/2759686324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581024d0 0x7fd658102950 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:25.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 -- 172.21.15.77:0/2759686324 >> 172.21.15.77:0/2759686324 conn(0x7fd658078490 msgr2=0x7fd6580788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:25.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 -- 172.21.15.77:0/2759686324 shutdown_connections 2024-05-27T08:36:25.085 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.083+0000 7fd65c93c700 1 -- 172.21.15.77:0/2759686324 wait complete. 2024-05-27T08:36:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65c93c700 1 Processor -- start 2024-05-27T08:36:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65c93c700 1 -- start start 2024-05-27T08:36:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65c93c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6581024d0 0x7fd6581040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65c93c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd658102e90 0x7fd6581045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65c93c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581084d0 0x7fd658104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65c93c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581126b0 con 0x7fd6581084d0 2024-05-27T08:36:25.087 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65c93c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd658112530 con 0x7fd658102e90 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65c93c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd658112830 con 0x7fd6581024d0 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd656d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581084d0 0x7fd658104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd658102e90 0x7fd6581045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd656d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581084d0 0x7fd658104b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51902/0 (socket says 172.21.15.77:51902) 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd656d9d700 1 -- 172.21.15.77:0/3107824679 learned_addr learned my addr 172.21.15.77:0/3107824679 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd65659c700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6581024d0 0x7fd6581040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd656d9d700 1 -- 172.21.15.77:0/3107824679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6581024d0 msgr2=0x7fd6581040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd656d9d700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6581024d0 0x7fd6581040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd656d9d700 1 -- 172.21.15.77:0/3107824679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd658102e90 msgr2=0x7fd6581045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd656d9d700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd658102e90 0x7fd6581045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd656d9d700 1 -- 172.21.15.77:0/3107824679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6581a8200 con 0x7fd6581084d0 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.084+0000 7fd655d9b700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd658102e90 0x7fd6581045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.085+0000 7fd656d9d700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581084d0 0x7fd658104b20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd65000b970 tx=0x7fd65000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:25.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.085+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650019070 con 0x7fd6581084d0 2024-05-27T08:36:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.085+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd65000cde0 con 0x7fd6581084d0 2024-05-27T08:36:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.085+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6581a84f0 con 0x7fd6581084d0 2024-05-27T08:36:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.085+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650007370 con 0x7fd6581084d0 2024-05-27T08:36:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.085+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6581a8a30 con 0x7fd6581084d0 2024-05-27T08:36:25.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.086+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd658063c50 con 0x7fd6581084d0 2024-05-27T08:36:25.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.087+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd65001c070 con 0x7fd6581084d0 2024-05-27T08:36:25.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.088+0000 7fd6477fe700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6400b2990 0x7fd6400b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:25.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.088+0000 7fd65659c700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6400b2990 0x7fd6400b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:25.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.088+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd6500e2180 con 0x7fd6581084d0 2024-05-27T08:36:25.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.088+0000 7fd6477fe700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] conn(0x7fd6400bc550 0x7fd6400be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:25.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.088+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 --> [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6400bf0c0 con 0x7fd6400bc550 2024-05-27T08:36:25.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.088+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fd6500e25b0 con 0x7fd6581084d0 2024-05-27T08:36:25.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.088+0000 7fd65659c700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6400b2990 0x7fd6400b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd648005ea0 tx=0x7fd648005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:25.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.088+0000 7fd655d9b700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] conn(0x7fd6400bc550 0x7fd6400be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:25.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.089+0000 7fd655d9b700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] conn(0x7fd6400bc550 0x7fd6400be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:25.091 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.089+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 <== osd.9 v2:172.21.15.77:6826/4049231537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd6400bf0c0 con 0x7fd6400bc550 2024-05-27T08:36:25.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 --> [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd658064050 con 0x7fd6400bc550 2024-05-27T08:36:25.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd6477fe700 1 -- 172.21.15.77:0/3107824679 <== osd.9 v2:172.21.15.77:6826/4049231537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd658064050 con 0x7fd6400bc550 2024-05-27T08:36:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] conn(0x7fd6400bc550 msgr2=0x7fd6400be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] conn(0x7fd6400bc550 0x7fd6400be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6400b2990 msgr2=0x7fd6400b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd6400b2990 0x7fd6400b4e50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd648005ea0 tx=0x7fd648005e30 comp rx=0 tx=0).stop 2024-05-27T08:36:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581084d0 msgr2=0x7fd658104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 --2- 172.21.15.77:0/3107824679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6581084d0 0x7fd658104b20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd65000b970 tx=0x7fd65000bc80 comp rx=0 tx=0).stop 2024-05-27T08:36:25.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd656d9d700 1 -- 172.21.15.77:0/3107824679 reap_dead start 2024-05-27T08:36:25.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 shutdown_connections 2024-05-27T08:36:25.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 >> 172.21.15.77:0/3107824679 conn(0x7fd658078490 msgr2=0x7fd65806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:25.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 shutdown_connections 2024-05-27T08:36:25.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:25.106+0000 7fd65c93c700 1 -- 172.21.15.77:0/3107824679 wait complete. 2024-05-27T08:36:25.784 INFO:teuthology.orchestra.run.smithi077.stdout:270582939689 2024-05-27T08:36:25.784 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-27T08:36:26.194 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:26.197 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:26.197 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:26.221 INFO:teuthology.orchestra.run.smithi077.stdout:244813135918 2024-05-27T08:36:26.222 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-27T08:36:26.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:26 smithi077 ceph-mon[32007]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-27T08:36:26.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:26 smithi179 ceph-mon[36481]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-27T08:36:26.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:26 smithi092 ceph-mon[36049]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-05-27T08:36:27.845 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2903449247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee541095b0 msgr2=0x7fee54109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 --2- 172.21.15.77:0/2903449247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee541095b0 0x7fee54109990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fee50009990 tx=0x7fee50009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2903449247 shutdown_connections 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 --2- 172.21.15.77:0/2903449247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee5406c650 0x7fee541103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 --2- 172.21.15.77:0/2903449247 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee541095b0 0x7fee54109990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 --2- 172.21.15.77:0/2903449247 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5406bc90 0x7fee5406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2903449247 >> 172.21.15.77:0/2903449247 conn(0x7fee54079410 msgr2=0x7fee54079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2903449247 shutdown_connections 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2903449247 wait complete. 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 Processor -- start 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.127+0000 7fee5b9d6700 1 -- start start 2024-05-27T08:36:27.846 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee5b9d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5406bc90 0x7fee54199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee5b9d6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee5406c650 0x7fee5419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee5b9d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee541095b0 0x7fee5419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee5b9d6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fee54113740 con 0x7fee5406c650 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee5b9d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fee541135c0 con 0x7fee5406bc90 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee5b9d6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee541138c0 con 0x7fee541095b0 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee5406c650 0x7fee5419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee5406c650 0x7fee5419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51914/0 (socket says 172.21.15.77:51914) 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 -- 172.21.15.77:0/2529493295 learned_addr learned my addr 172.21.15.77:0/2529493295 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee59f73700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee541095b0 0x7fee5419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee59772700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5406bc90 0x7fee54199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 -- 172.21.15.77:0/2529493295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee541095b0 msgr2=0x7fee5419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee541095b0 0x7fee5419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 -- 172.21.15.77:0/2529493295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5406bc90 msgr2=0x7fee54199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fee5406bc90 0x7fee54199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 -- 172.21.15.77:0/2529493295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5419ed20 con 0x7fee5406c650 2024-05-27T08:36:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee59f73700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee541095b0 0x7fee5419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:36:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee58f71700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee5406c650 0x7fee5419a250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fee4400d950 tx=0x7fee4400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44016070 con 0x7fee5406c650 2024-05-27T08:36:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5419f010 con 0x7fee5406c650 2024-05-27T08:36:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee44010460 con 0x7fee5406c650 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4400f620 con 0x7fee5406c650 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.128+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee541a6890 con 0x7fee5406c650 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.129+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee54063c50 con 0x7fee5406c650 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.130+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fee44019070 con 0x7fee5406c650 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.131+0000 7fee4a7fc700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fee400b2910 0x7fee400b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.131+0000 7fee59772700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fee400b2910 0x7fee400b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.131+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fee440e2560 con 0x7fee5406c650 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.131+0000 7fee4a7fc700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] conn(0x7fee400bc4d0 0x7fee400be970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.131+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 --> [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee400bf040 con 0x7fee400bc4d0 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.131+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fee440e2990 con 0x7fee5406c650 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.132+0000 7fee59772700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fee400b2910 0x7fee400b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee50005cd0 tx=0x7fee50005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.132+0000 7fee59f73700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] conn(0x7fee400bc4d0 0x7fee400be970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.132+0000 7fee59f73700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] conn(0x7fee400bc4d0 0x7fee400be970 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.133+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 <== osd.6 v2:172.21.15.77:6818/3753560520 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee400bf040 con 0x7fee400bc4d0 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.149+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 --> [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee540640d0 con 0x7fee400bc4d0 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.149+0000 7fee4a7fc700 1 -- 172.21.15.77:0/2529493295 <== osd.6 v2:172.21.15.77:6818/3753560520 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fee540640d0 con 0x7fee400bc4d0 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.149+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] conn(0x7fee400bc4d0 msgr2=0x7fee400be970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.149+0000 7fee5b9d6700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] conn(0x7fee400bc4d0 0x7fee400be970 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fee400b2910 msgr2=0x7fee400b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee5b9d6700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fee400b2910 0x7fee400b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee50005cd0 tx=0x7fee50005c40 comp rx=0 tx=0).stop 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee5406c650 msgr2=0x7fee5419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee5b9d6700 1 --2- 172.21.15.77:0/2529493295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee5406c650 0x7fee5419a250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fee4400d950 tx=0x7fee4400dd10 comp rx=0 tx=0).stop 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee59f73700 1 -- 172.21.15.77:0/2529493295 reap_dead start 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 shutdown_connections 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 >> 172.21.15.77:0/2529493295 conn(0x7fee54079410 msgr2=0x7fee541011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 shutdown_connections 2024-05-27T08:36:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:27.150+0000 7fee5b9d6700 1 -- 172.21.15.77:0/2529493295 wait complete. 2024-05-27T08:36:28.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.300+0000 7f168edf1700 1 -- 172.21.15.77:0/2777651672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 msgr2=0x7f1688103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:28.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.300+0000 7f168edf1700 1 --2- 172.21.15.77:0/2777651672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 0x7f1688103c00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f167c0099c0 tx=0x7f167c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.300+0000 7f168edf1700 1 -- 172.21.15.77:0/2777651672 shutdown_connections 2024-05-27T08:36:28.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.300+0000 7f168edf1700 1 --2- 172.21.15.77:0/2777651672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1688108910 0x7f1688110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.300+0000 7f168edf1700 1 --2- 172.21.15.77:0/2777651672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 0x7f1688103c00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.300+0000 7f168edf1700 1 --2- 172.21.15.77:0/2777651672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16881041d0 0x7f16881081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.300+0000 7f168edf1700 1 -- 172.21.15.77:0/2777651672 >> 172.21.15.77:0/2777651672 conn(0x7f168807a6b0 msgr2=0x7f168807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:28.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.300+0000 7f168edf1700 1 -- 172.21.15.77:0/2777651672 shutdown_connections 2024-05-27T08:36:28.303 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 -- 172.21.15.77:0/2777651672 wait complete. 2024-05-27T08:36:28.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 Processor -- start 2024-05-27T08:36:28.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 -- start start 2024-05-27T08:36:28.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 0x7f168819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:28.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16881041d0 0x7f168819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:28.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1688108910 0x7f1688199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:28.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1688113760 con 0x7f1688103820 2024-05-27T08:36:28.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f16881135e0 con 0x7f16881041d0 2024-05-27T08:36:28.305 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168edf1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16881138e0 con 0x7f1688108910 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.301+0000 7f168cb8d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 0x7f168819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168cb8d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 0x7f168819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51922/0 (socket says 172.21.15.77:51922) 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168cb8d700 1 -- 172.21.15.77:0/3317242621 learned_addr learned my addr 172.21.15.77:0/3317242621 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168d38e700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1688108910 0x7f1688199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f1687fff700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16881041d0 0x7f168819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168cb8d700 1 -- 172.21.15.77:0/3317242621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1688108910 msgr2=0x7f1688199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168cb8d700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1688108910 0x7f1688199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168cb8d700 1 -- 172.21.15.77:0/3317242621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16881041d0 msgr2=0x7f168819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168cb8d700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16881041d0 0x7f168819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.306 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168cb8d700 1 -- 172.21.15.77:0/3317242621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168819a1c0 con 0x7f1688103820 2024-05-27T08:36:28.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168cb8d700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 0x7f168819f850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:28.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c026070 con 0x7f1688103820 2024-05-27T08:36:28.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167c003f20 con 0x7f1688103820 2024-05-27T08:36:28.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c00fc00 con 0x7f1688103820 2024-05-27T08:36:28.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.302+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168819a450 con 0x7f1688103820 2024-05-27T08:36:28.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.303+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16881a6720 con 0x7f1688103820 2024-05-27T08:36:28.307 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.303+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1688063c90 con 0x7f1688103820 2024-05-27T08:36:28.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.305+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f167c029070 con 0x7f1688103820 2024-05-27T08:36:28.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.305+0000 7f1685ffb700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16780b2990 0x7f16780b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:28.308 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.305+0000 7f1687fff700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16780b2990 0x7f16780b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:28.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.306+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f167c0e2fa0 con 0x7f1688103820 2024-05-27T08:36:28.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.306+0000 7f1685ffb700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] conn(0x7f16780bc550 0x7f16780be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:28.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.306+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 --> [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f167c00eea0 con 0x7f16780bc550 2024-05-27T08:36:28.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.306+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f167c0e3480 con 0x7f1688103820 2024-05-27T08:36:28.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.306+0000 7f1687fff700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16780b2990 0x7f16780b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1688109d30 tx=0x7f167400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:28.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.306+0000 7f168d38e700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] conn(0x7f16780bc550 0x7f16780be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:28.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.306+0000 7f168d38e700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] conn(0x7f16780bc550 0x7f16780be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:28.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.307+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 <== osd.5 v2:172.21.15.179:6808/1761235705 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f167c00eea0 con 0x7f16780bc550 2024-05-27T08:36:28.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.324+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 --> [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1688064160 con 0x7f16780bc550 2024-05-27T08:36:28.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.324+0000 7f1685ffb700 1 -- 172.21.15.77:0/3317242621 <== osd.5 v2:172.21.15.179:6808/1761235705 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1688064160 con 0x7f16780bc550 2024-05-27T08:36:28.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 >> [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] conn(0x7f16780bc550 msgr2=0x7f16780be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] conn(0x7f16780bc550 0x7f16780be9f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16780b2990 msgr2=0x7f16780b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16780b2990 0x7f16780b4e50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1688109d30 tx=0x7f167400b3c0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 msgr2=0x7f168819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 --2- 172.21.15.77:0/3317242621 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1688103820 0x7f168819f850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c0047b0 comp rx=0 tx=0).stop 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168d38e700 1 -- 172.21.15.77:0/3317242621 reap_dead start 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 shutdown_connections 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 >> 172.21.15.77:0/3317242621 conn(0x7f168807a6b0 msgr2=0x7f1688102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 shutdown_connections 2024-05-27T08:36:28.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:28.325+0000 7f168edf1700 1 -- 172.21.15.77:0/3317242621 wait complete. 2024-05-27T08:36:28.685 INFO:teuthology.orchestra.run.smithi077.stdout:197568495672 2024-05-27T08:36:28.686 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-27T08:36:28.709 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:28 smithi077 ceph-mon[32007]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T08:36:28.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:28 smithi179 ceph-mon[36481]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T08:36:28.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:28 smithi092 ceph-mon[36049]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T08:36:29.533 INFO:teuthology.orchestra.run.smithi077.stdout:171798691902 2024-05-27T08:36:29.534 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-27T08:36:30.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:30 smithi077 ceph-mon[32007]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:30.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:30 smithi179 ceph-mon[36481]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:30.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:30 smithi092 ceph-mon[36049]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:31.436 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:31.439 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:31.447 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:32.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:32 smithi179 ceph-mon[36481]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:32.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:32 smithi077 ceph-mon[32007]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:32.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:32 smithi092 ceph-mon[36049]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:34.587 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:34.587 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:34.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:34 smithi179 ceph-mon[36481]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:34.868 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:34.869 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:34.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:34 smithi077 ceph-mon[32007]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:34.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:34 smithi092 ceph-mon[36049]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.401+0000 7f6239c0a700 1 -- 172.21.15.77:0/1951199654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102df0 msgr2=0x7f623410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:35.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.401+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1951199654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102df0 0x7f623410f2e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b0a0 tx=0x7f622c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.401+0000 7f6239c0a700 1 -- 172.21.15.77:0/1951199654 shutdown_connections 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.401+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1951199654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102df0 0x7f623410f2e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.401+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1951199654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6234108510 0x7f62341088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.401+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1951199654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6234102430 0x7f62341028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.402+0000 7f6239c0a700 1 -- 172.21.15.77:0/1951199654 >> 172.21.15.77:0/1951199654 conn(0x7f62340793b0 msgr2=0x7f62340797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.402+0000 7f6239c0a700 1 -- 172.21.15.77:0/1951199654 shutdown_connections 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.402+0000 7f6239c0a700 1 -- 172.21.15.77:0/1951199654 wait complete. 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.402+0000 7f6239c0a700 1 Processor -- start 2024-05-27T08:36:35.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.402+0000 7f6239c0a700 1 -- start start 2024-05-27T08:36:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6239c0a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102430 0x7f623419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6239c0a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6234102df0 0x7f623419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6239c0a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6234108510 0x7f6234198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6239c0a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6234112660 con 0x7f6234102430 2024-05-27T08:36:35.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6239c0a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f62341124e0 con 0x7f6234102df0 2024-05-27T08:36:35.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6239c0a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f62341127e0 con 0x7f6234108510 2024-05-27T08:36:35.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102430 0x7f623419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:35.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102430 0x7f623419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48200/0 (socket says 172.21.15.77:48200) 2024-05-27T08:36:35.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 -- 172.21.15.77:0/1393787430 learned_addr learned my addr 172.21.15.77:0/1393787430 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:35.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6232ffd700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6234102df0 0x7f623419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:35.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6233fff700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6234108510 0x7f6234198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 -- 172.21.15.77:0/1393787430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6234108510 msgr2=0x7f6234198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6234108510 0x7f6234198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 -- 172.21.15.77:0/1393787430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6234102df0 msgr2=0x7f623419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6234102df0 0x7f623419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 -- 172.21.15.77:0/1393787430 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62341990e0 con 0x7f6234102430 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f62337fe700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102430 0x7f623419e7c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f622400b530 tx=0x7f622400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6230ff9700 1 -- 172.21.15.77:0/1393787430 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224015020 con 0x7f6234102430 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6230ff9700 1 -- 172.21.15.77:0/1393787430 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6224011ce0 con 0x7f6234102430 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6230ff9700 1 -- 172.21.15.77:0/1393787430 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6224010dd0 con 0x7f6234102430 2024-05-27T08:36:35.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.403+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62341993d0 con 0x7f6234102430 2024-05-27T08:36:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.404+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62341a5690 con 0x7f6234102430 2024-05-27T08:36:35.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.404+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623410a9a0 con 0x7f6234102430 2024-05-27T08:36:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.406+0000 7f6230ff9700 1 -- 172.21.15.77:0/1393787430 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6224018020 con 0x7f6234102430 2024-05-27T08:36:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.406+0000 7f6230ff9700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f621c0b2910 0x7f621c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.406+0000 7f6232ffd700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f621c0b2910 0x7f621c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.407+0000 7f6230ff9700 1 -- 172.21.15.77:0/1393787430 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f62240e1f10 con 0x7f6234102430 2024-05-27T08:36:35.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.407+0000 7f6232ffd700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f621c0b2910 0x7f621c0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6234079a20 tx=0x7f622800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:35.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.410+0000 7f6230ff9700 1 -- 172.21.15.77:0/1393787430 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f622401f020 con 0x7f6234102430 2024-05-27T08:36:35.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.628+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f623406a230 con 0x7f6234102430 2024-05-27T08:36:35.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.629+0000 7f6230ff9700 1 -- 172.21.15.77:0/1393787430 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f62240a9be0 con 0x7f6234102430 2024-05-27T08:36:35.631 INFO:teuthology.orchestra.run.smithi077.stdout:55834574931 2024-05-27T08:36:35.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f621c0b2910 msgr2=0x7f621c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f621c0b2910 0x7f621c0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6234079a20 tx=0x7f622800b3c0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102430 msgr2=0x7f623419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102430 0x7f623419e7c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f622400b530 tx=0x7f622400b840 comp rx=0 tx=0).stop 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 shutdown_connections 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f621c0b2910 0x7f621c0b4dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6234108510 0x7f6234198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6234102430 0x7f623419e7c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 --2- 172.21.15.77:0/1393787430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6234102df0 0x7f623419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 >> 172.21.15.77:0/1393787430 conn(0x7f62340793b0 msgr2=0x7f6234100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 shutdown_connections 2024-05-27T08:36:35.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.631+0000 7f6239c0a700 1 -- 172.21.15.77:0/1393787430 wait complete. 2024-05-27T08:36:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 -- 172.21.15.77:0/2695152243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 msgr2=0x7f42b410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 --2- 172.21.15.77:0/2695152243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 0x7f42b410f2e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00b0a0 tx=0x7f42ac00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 -- 172.21.15.77:0/2695152243 shutdown_connections 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 --2- 172.21.15.77:0/2695152243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 0x7f42b410f2e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 --2- 172.21.15.77:0/2695152243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b4108510 0x7f42b41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 --2- 172.21.15.77:0/2695152243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42b4102430 0x7f42b41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 -- 172.21.15.77:0/2695152243 >> 172.21.15.77:0/2695152243 conn(0x7f42b40793b0 msgr2=0x7f42b40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 -- 172.21.15.77:0/2695152243 shutdown_connections 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.951+0000 7f42bad5d700 1 -- 172.21.15.77:0/2695152243 wait complete. 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42bad5d700 1 Processor -- start 2024-05-27T08:36:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42bad5d700 1 -- start start 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42bad5d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b4102430 0x7f42b4104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42bad5d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 0x7f42b41045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42bad5d700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42b4108510 0x7f42b4104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42bad5d700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42b41126c0 con 0x7f42b4102df0 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42bad5d700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f42b4112540 con 0x7f42b4108510 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42bad5d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f42b4112840 con 0x7f42b4102430 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42b3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 0x7f42b41045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42b3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 0x7f42b41045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48224/0 (socket says 172.21.15.77:48224) 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42b3fff700 1 -- 172.21.15.77:0/1246934281 learned_addr learned my addr 172.21.15.77:0/1246934281 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42b92fa700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42b4108510 0x7f42b4104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.952+0000 7f42b8af9700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b4102430 0x7f42b4104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b3fff700 1 -- 172.21.15.77:0/1246934281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b4102430 msgr2=0x7f42b4104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b3fff700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b4102430 0x7f42b4104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b3fff700 1 -- 172.21.15.77:0/1246934281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42b4108510 msgr2=0x7f42b4104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b3fff700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42b4108510 0x7f42b4104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b3fff700 1 -- 172.21.15.77:0/1246934281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b41a8230 con 0x7f42b4102df0 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b3fff700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 0x7f42b41045b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f42a000b530 tx=0x7f42a000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b1ffb700 1 -- 172.21.15.77:0/1246934281 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0015020 con 0x7f42b4102df0 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b41a8520 con 0x7f42b4102df0 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b1ffb700 1 -- 172.21.15.77:0/1246934281 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42a0011ce0 con 0x7f42b4102df0 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42b1ffb700 1 -- 172.21.15.77:0/1246934281 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a0010dd0 con 0x7f42b4102df0 2024-05-27T08:36:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.953+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b41a8ab0 con 0x7f42b4102df0 2024-05-27T08:36:35.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.954+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b406a230 con 0x7f42b4102df0 2024-05-27T08:36:35.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.956+0000 7f42b1ffb700 1 -- 172.21.15.77:0/1246934281 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f42a0018020 con 0x7f42b4102df0 2024-05-27T08:36:35.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.956+0000 7f42b1ffb700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42a40b2910 0x7f42a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:35.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.956+0000 7f42b8af9700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42a40b2910 0x7f42a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:35.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.956+0000 7f42b1ffb700 1 -- 172.21.15.77:0/1246934281 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f42a00e1f10 con 0x7f42b4102df0 2024-05-27T08:36:35.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.957+0000 7f42b8af9700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42a40b2910 0x7f42a40b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009a10 tx=0x7f42a8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:35.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:35.960+0000 7f42b1ffb700 1 -- 172.21.15.77:0/1246934281 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f42a001f020 con 0x7f42b4102df0 2024-05-27T08:36:36.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.174+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f42b402e560 con 0x7f42b4102df0 2024-05-27T08:36:36.176 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.175+0000 7f42b1ffb700 1 -- 172.21.15.77:0/1246934281 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f42a00a9be0 con 0x7f42b4102df0 2024-05-27T08:36:36.176 INFO:teuthology.orchestra.run.smithi077.stdout:141733920835 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42a40b2910 msgr2=0x7f42a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42a40b2910 0x7f42a40b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009a10 tx=0x7f42a8009190 comp rx=0 tx=0).stop 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 msgr2=0x7f42b41045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 0x7f42b41045b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f42a000b530 tx=0x7f42a000b840 comp rx=0 tx=0).stop 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 shutdown_connections 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42b4108510 0x7f42b4104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f42a40b2910 0x7f42a40b4dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42b4102430 0x7f42b4104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 --2- 172.21.15.77:0/1246934281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42b4102df0 0x7f42b41045b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 >> 172.21.15.77:0/1246934281 conn(0x7f42b40793b0 msgr2=0x7f42b406be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 shutdown_connections 2024-05-27T08:36:36.179 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:36.177+0000 7f42bad5d700 1 -- 172.21.15.77:0/1246934281 wait complete. 2024-05-27T08:36:36.797 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:36 smithi077 ceph-mon[32007]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:36.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:36 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1393787430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T08:36:36.798 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:36 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1246934281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T08:36:36.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:36 smithi179 ceph-mon[36481]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:36.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:36 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1393787430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T08:36:36.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:36 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1246934281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T08:36:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:36 smithi092 ceph-mon[36049]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:36 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1393787430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-27T08:36:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:36 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1246934281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-27T08:36:37.141 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574931 for osd.1 2024-05-27T08:36:37.142 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:37.566 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:38.810 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:38 smithi077 ceph-mon[32007]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-27T08:36:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 -- 172.21.15.77:0/1157823371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7448108620 msgr2=0x7f7448108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1157823371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7448108620 0x7f7448108a00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7438009990 tx=0x7f7438009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 -- 172.21.15.77:0/1157823371 shutdown_connections 2024-05-27T08:36:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1157823371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74481028c0 0x7f7448110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1157823371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7448108620 0x7f7448108a00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1157823371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7448101f00 0x7f7448102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 -- 172.21.15.77:0/1157823371 >> 172.21.15.77:0/1157823371 conn(0x7f7448077780 msgr2=0x7f7448077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:38.818 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 -- 172.21.15.77:0/1157823371 shutdown_connections 2024-05-27T08:36:38.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.816+0000 7f744d1c1700 1 -- 172.21.15.77:0/1157823371 wait complete. 2024-05-27T08:36:38.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.817+0000 7f744d1c1700 1 Processor -- start 2024-05-27T08:36:38.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744d1c1700 1 -- start start 2024-05-27T08:36:38.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744d1c1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7448101f00 0x7f744819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744d1c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481028c0 0x7f744819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744d1c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448108620 0x7f74481999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744d1c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74481137b0 con 0x7f74481028c0 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744d1c1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7448113630 con 0x7f7448101f00 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744d1c1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7448113930 con 0x7f7448108620 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481028c0 0x7f744819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481028c0 0x7f744819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48240/0 (socket says 172.21.15.77:48240) 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 -- 172.21.15.77:0/1526689046 learned_addr learned my addr 172.21.15.77:0/1526689046 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f7446d9d700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7448101f00 0x7f744819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744759e700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448108620 0x7f74481999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 -- 172.21.15.77:0/1526689046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448108620 msgr2=0x7f74481999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:38.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448108620 0x7f74481999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 -- 172.21.15.77:0/1526689046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7448101f00 msgr2=0x7f744819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7448101f00 0x7f744819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 -- 172.21.15.77:0/1526689046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744819a150 con 0x7f74481028c0 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.818+0000 7f744659c700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481028c0 0x7f744819fd70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f743c00d950 tx=0x7f743c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.819+0000 7f742ffff700 1 -- 172.21.15.77:0/1526689046 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c016070 con 0x7f74481028c0 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.819+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744819a440 con 0x7f74481028c0 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.819+0000 7f742ffff700 1 -- 172.21.15.77:0/1526689046 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f743c009df0 con 0x7f74481028c0 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.819+0000 7f742ffff700 1 -- 172.21.15.77:0/1526689046 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c00f3c0 con 0x7f74481028c0 2024-05-27T08:36:38.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.819+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74481a6780 con 0x7f74481028c0 2024-05-27T08:36:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.820+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f744806a230 con 0x7f74481028c0 2024-05-27T08:36:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.821+0000 7f742ffff700 1 -- 172.21.15.77:0/1526689046 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f743c019070 con 0x7f74481028c0 2024-05-27T08:36:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.822+0000 7f742ffff700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74300b2910 0x7f74300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.822+0000 7f7446d9d700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74300b2910 0x7f74300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:38.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.822+0000 7f742ffff700 1 -- 172.21.15.77:0/1526689046 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f743c0e1fc0 con 0x7f74481028c0 2024-05-27T08:36:38.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.822+0000 7f7446d9d700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74300b2910 0x7f74300b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7438006010 tx=0x7f743800b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:38.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:38.826+0000 7f742ffff700 1 -- 172.21.15.77:0/1526689046 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f743c0a9b80 con 0x7f74481028c0 2024-05-27T08:36:38.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:38 smithi179 ceph-mon[36481]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-27T08:36:38.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:38 smithi092 ceph-mon[36049]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-05-27T08:36:39.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.038+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f744802e560 con 0x7f74481028c0 2024-05-27T08:36:39.040 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.038+0000 7f742ffff700 1 -- 172.21.15.77:0/1526689046 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f743c0a9480 con 0x7f74481028c0 2024-05-27T08:36:39.040 INFO:teuthology.orchestra.run.smithi077.stdout:77309411406 2024-05-27T08:36:39.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74300b2910 msgr2=0x7f74300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:39.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74300b2910 0x7f74300b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7438006010 tx=0x7f743800b560 comp rx=0 tx=0).stop 2024-05-27T08:36:39.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481028c0 msgr2=0x7f744819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:39.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481028c0 0x7f744819fd70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f743c00d950 tx=0x7f743c00dc60 comp rx=0 tx=0).stop 2024-05-27T08:36:39.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 shutdown_connections 2024-05-27T08:36:39.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f74300b2910 0x7f74300b4dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.043 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448108620 0x7f74481999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7448101f00 0x7f744819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 --2- 172.21.15.77:0/1526689046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74481028c0 0x7f744819fd70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.041+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 >> 172.21.15.77:0/1526689046 conn(0x7f7448077780 msgr2=0x7f7448100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:39.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.042+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 shutdown_connections 2024-05-27T08:36:39.044 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.042+0000 7f744d1c1700 1 -- 172.21.15.77:0/1526689046 wait complete. 2024-05-27T08:36:39.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 -- 172.21.15.77:0/2319583250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84108520 msgr2=0x7fca84108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:39.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 --2- 172.21.15.77:0/2319583250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84108520 0x7fca84108900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fca74009990 tx=0x7fca74009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 -- 172.21.15.77:0/2319583250 shutdown_connections 2024-05-27T08:36:39.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 --2- 172.21.15.77:0/2319583250 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca84102e00 0x7fca8410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 --2- 172.21.15.77:0/2319583250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84108520 0x7fca84108900 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 --2- 172.21.15.77:0/2319583250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca84102440 0x7fca841028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 -- 172.21.15.77:0/2319583250 >> 172.21.15.77:0/2319583250 conn(0x7fca840793f0 msgr2=0x7fca84079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:39.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 -- 172.21.15.77:0/2319583250 shutdown_connections 2024-05-27T08:36:39.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.226+0000 7fca88d91700 1 -- 172.21.15.77:0/2319583250 wait complete. 2024-05-27T08:36:39.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca88d91700 1 Processor -- start 2024-05-27T08:36:39.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca88d91700 1 -- start start 2024-05-27T08:36:39.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca88d91700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca84102440 0x7fca841044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:39.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca88d91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84102e00 0x7fca84104a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca88d91700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca84108520 0x7fca84104f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca88d91700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca84112640 con 0x7fca84102e00 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca88d91700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca841124c0 con 0x7fca84108520 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca88d91700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca841127c0 con 0x7fca84102440 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca81d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84102e00 0x7fca84104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca81d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84102e00 0x7fca84104a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48252/0 (socket says 172.21.15.77:48252) 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca81d9b700 1 -- 172.21.15.77:0/3667121123 learned_addr learned my addr 172.21.15.77:0/3667121123 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca8259c700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca84102440 0x7fca841044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca82d9d700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca84108520 0x7fca84104f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca81d9b700 1 -- 172.21.15.77:0/3667121123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca84102440 msgr2=0x7fca841044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:39.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca81d9b700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca84102440 0x7fca841044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca81d9b700 1 -- 172.21.15.77:0/3667121123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca84108520 msgr2=0x7fca84104f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca81d9b700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca84108520 0x7fca84104f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca81d9b700 1 -- 172.21.15.77:0/3667121123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca841a81a0 con 0x7fca84102e00 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.227+0000 7fca8259c700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca84102440 0x7fca841044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.228+0000 7fca81d9b700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84102e00 0x7fca84104a10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca7800d980 tx=0x7fca7800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.228+0000 7fca737fe700 1 -- 172.21.15.77:0/3667121123 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78017070 con 0x7fca84102e00 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.228+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca841a8490 con 0x7fca84102e00 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.228+0000 7fca737fe700 1 -- 172.21.15.77:0/3667121123 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca78004830 con 0x7fca84102e00 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.228+0000 7fca737fe700 1 -- 172.21.15.77:0/3667121123 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78005440 con 0x7fca84102e00 2024-05-27T08:36:39.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.228+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca841a8980 con 0x7fca84102e00 2024-05-27T08:36:39.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.229+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8406a230 con 0x7fca84102e00 2024-05-27T08:36:39.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.230+0000 7fca737fe700 1 -- 172.21.15.77:0/3667121123 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fca780055e0 con 0x7fca84102e00 2024-05-27T08:36:39.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.231+0000 7fca737fe700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fca6c0b2910 0x7fca6c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:39.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.231+0000 7fca8259c700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fca6c0b2910 0x7fca6c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:39.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.231+0000 7fca737fe700 1 -- 172.21.15.77:0/3667121123 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fca780e2350 con 0x7fca84102e00 2024-05-27T08:36:39.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.231+0000 7fca8259c700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fca6c0b2910 0x7fca6c0b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca74006010 tx=0x7fca7401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:39.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.238+0000 7fca737fe700 1 -- 172.21.15.77:0/3667121123 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca78020020 con 0x7fca84102e00 2024-05-27T08:36:39.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.456+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fca841a8be0 con 0x7fca84102e00 2024-05-27T08:36:39.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.456+0000 7fca737fe700 1 -- 172.21.15.77:0/3667121123 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fca780a9f10 con 0x7fca84102e00 2024-05-27T08:36:39.458 INFO:teuthology.orchestra.run.smithi077.stdout:322122547233 2024-05-27T08:36:39.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fca6c0b2910 msgr2=0x7fca6c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fca6c0b2910 0x7fca6c0b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fca74006010 tx=0x7fca7401a040 comp rx=0 tx=0).stop 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84102e00 msgr2=0x7fca84104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84102e00 0x7fca84104a10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca7800d980 tx=0x7fca7800dd40 comp rx=0 tx=0).stop 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 shutdown_connections 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca84108520 0x7fca84104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fca6c0b2910 0x7fca6c0b4dd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca84102440 0x7fca841044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 --2- 172.21.15.77:0/3667121123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca84102e00 0x7fca84104a10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 >> 172.21.15.77:0/3667121123 conn(0x7fca840793f0 msgr2=0x7fca841068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 shutdown_connections 2024-05-27T08:36:39.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:39.459+0000 7fca88d91700 1 -- 172.21.15.77:0/3667121123 wait complete. 2024-05-27T08:36:39.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:39 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1526689046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T08:36:39.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:39 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3667121123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T08:36:39.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:39 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1526689046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T08:36:39.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:39 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3667121123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T08:36:39.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:39 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1526689046' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-27T08:36:39.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:39 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3667121123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-27T08:36:40.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:40 smithi077 ceph-mon[32007]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-27T08:36:40.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:40 smithi179 ceph-mon[36481]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-27T08:36:40.886 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:40 smithi092 ceph-mon[36049]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-05-27T08:36:41.104 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547229 got 322122547233 for osd.11 2024-05-27T08:36:41.104 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:41.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920832 got 141733920835 for osd.4 2024-05-27T08:36:41.503 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:42.071 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411403 got 77309411406 for osd.2 2024-05-27T08:36:42.072 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:42.701 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:42 smithi077 ceph-mon[32007]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:42.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:42 smithi179 ceph-mon[36481]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:42.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:42 smithi092 ceph-mon[36049]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:44.654 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:44 smithi077 ceph-mon[32007]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:44.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:44 smithi179 ceph-mon[36481]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:44.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:44 smithi092 ceph-mon[36049]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 -- 172.21.15.77:0/926947072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 msgr2=0x7f5b6c110490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:44.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 --2- 172.21.15.77:0/926947072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 0x7f5b6c110490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400b0d0 tx=0x7f5b6400b3e0 comp rx=0 tx=0).stop 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 -- 172.21.15.77:0/926947072 shutdown_connections 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 --2- 172.21.15.77:0/926947072 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 0x7f5b6c110490 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 --2- 172.21.15.77:0/926947072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b6c103820 0x7f5b6c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 --2- 172.21.15.77:0/926947072 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6c1041d0 0x7f5b6c1081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 -- 172.21.15.77:0/926947072 >> 172.21.15.77:0/926947072 conn(0x7f5b6c07a6b0 msgr2=0x7f5b6c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 -- 172.21.15.77:0/926947072 shutdown_connections 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 -- 172.21.15.77:0/926947072 wait complete. 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 Processor -- start 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.828+0000 7f5b714ca700 1 -- start start 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b714ca700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6c103820 0x7f5b6c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b714ca700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b6c1041d0 0x7f5b6c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:44.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b714ca700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 0x7f5b6c199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b714ca700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1136e0 con 0x7f5b6c108910 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b714ca700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c113560 con 0x7f5b6c103820 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b714ca700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c113860 con 0x7f5b6c1041d0 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 0x7f5b6c199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 0x7f5b6c199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45126/0 (socket says 172.21.15.77:45126) 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 -- 172.21.15.77:0/682895064 learned_addr learned my addr 172.21.15.77:0/682895064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6a7fc700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b6c1041d0 0x7f5b6c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6affd700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6c103820 0x7f5b6c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 -- 172.21.15.77:0/682895064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b6c1041d0 msgr2=0x7f5b6c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b6c1041d0 0x7f5b6c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 -- 172.21.15.77:0/682895064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6c103820 msgr2=0x7f5b6c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6c103820 0x7f5b6c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 -- 172.21.15.77:0/682895064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c199fe0 con 0x7f5b6c108910 2024-05-27T08:36:44.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6affd700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6c103820 0x7f5b6c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b6b7fe700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 0x7f5b6c199880 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b640095c0 tx=0x7f5b64007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b53fff700 1 -- 172.21.15.77:0/682895064 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6401f070 con 0x7f5b6c108910 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b53fff700 1 -- 172.21.15.77:0/682895064 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b64003c00 con 0x7f5b6c108910 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c19a270 con 0x7f5b6c108910 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b53fff700 1 -- 172.21.15.77:0/682895064 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6401b6a0 con 0x7f5b6c108910 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.829+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1a6590 con 0x7f5b6c108910 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.831+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c104ca0 con 0x7f5b6c108910 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.832+0000 7f5b53fff700 1 -- 172.21.15.77:0/682895064 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5b64022070 con 0x7f5b6c108910 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.833+0000 7f5b53fff700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b540b2910 0x7f5b540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.833+0000 7f5b6affd700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b540b2910 0x7f5b540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.833+0000 7f5b53fff700 1 -- 172.21.15.77:0/682895064 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f5b6400f020 con 0x7f5b6c108910 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.833+0000 7f5b6affd700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b540b2910 0x7f5b540b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005fd0 tx=0x7f5b5c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:44.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:44.840+0000 7f5b53fff700 1 -- 172.21.15.77:0/682895064 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b64029020 con 0x7f5b6c108910 2024-05-27T08:36:45.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.060+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5b6c1a6760 con 0x7f5b6c108910 2024-05-27T08:36:45.062 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.060+0000 7f5b53fff700 1 -- 172.21.15.77:0/682895064 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5b640ab640 con 0x7f5b6c108910 2024-05-27T08:36:45.063 INFO:teuthology.orchestra.run.smithi077.stdout:244813135922 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b540b2910 msgr2=0x7f5b540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b540b2910 0x7f5b540b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005fd0 tx=0x7f5b5c005f00 comp rx=0 tx=0).stop 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 msgr2=0x7f5b6c199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 0x7f5b6c199880 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5b640095c0 tx=0x7f5b64007a20 comp rx=0 tx=0).stop 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 shutdown_connections 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6c108910 0x7f5b6c199880 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b6c103820 0x7f5b6c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f5b540b2910 0x7f5b540b4dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 --2- 172.21.15.77:0/682895064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b6c1041d0 0x7f5b6c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 >> 172.21.15.77:0/682895064 conn(0x7f5b6c07a6b0 msgr2=0x7f5b6c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.062+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 shutdown_connections 2024-05-27T08:36:45.065 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:45.063+0000 7f5b714ca700 1 -- 172.21.15.77:0/682895064 wait complete. 2024-05-27T08:36:45.706 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:45 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/682895064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T08:36:45.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:45 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/682895064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T08:36:45.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:45 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/682895064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-27T08:36:46.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- 172.21.15.77:0/1386712041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f8410ff40 msgr2=0x7f7f84112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:46.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 --2- 172.21.15.77:0/1386712041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f8410ff40 0x7f7f84112330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00b0a0 tx=0x7f7f7c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- 172.21.15.77:0/1386712041 shutdown_connections 2024-05-27T08:36:46.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 --2- 172.21.15.77:0/1386712041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f8410ff40 0x7f7f84112330 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 --2- 172.21.15.77:0/1386712041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f84075770 0x7f7f84075b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 --2- 172.21.15.77:0/1386712041 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f84076090 0x7f7f8410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- 172.21.15.77:0/1386712041 >> 172.21.15.77:0/1386712041 conn(0x7f7f8407a6a0 msgr2=0x7f7f8407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- 172.21.15.77:0/1386712041 shutdown_connections 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- 172.21.15.77:0/1386712041 wait complete. 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 Processor -- start 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- start start 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84075770 0x7f7f8419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f84076090 0x7f7f8419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f8410ff40 0x7f7f841a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84115940 con 0x7f7f84075770 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7f841157c0 con 0x7f7f84076090 2024-05-27T08:36:46.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.126+0000 7f7f8bece700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7f84115ac0 con 0x7f7f8410ff40 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84075770 0x7f7f8419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84075770 0x7f7f8419bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45148/0 (socket says 172.21.15.77:45148) 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 -- 172.21.15.77:0/520335017 learned_addr learned my addr 172.21.15.77:0/520335017 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89469700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f84076090 0x7f7f8419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f8a46b700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f8410ff40 0x7f7f841a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 -- 172.21.15.77:0/520335017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f8410ff40 msgr2=0x7f7f841a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f8410ff40 0x7f7f841a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 -- 172.21.15.77:0/520335017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f84076090 msgr2=0x7f7f8419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f84076090 0x7f7f8419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 -- 172.21.15.77:0/520335017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f841a0d70 con 0x7f7f84075770 2024-05-27T08:36:46.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f8a46b700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f8410ff40 0x7f7f841a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f89c6a700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84075770 0x7f7f8419bdb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000b530 tx=0x7f7f8000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f7affd700 1 -- 172.21.15.77:0/520335017 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80015020 con 0x7f7f84075770 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f841a1060 con 0x7f7f84075770 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f7affd700 1 -- 172.21.15.77:0/520335017 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f80011ce0 con 0x7f7f84075770 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f7affd700 1 -- 172.21.15.77:0/520335017 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80010e40 con 0x7f7f84075770 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.127+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f841a88e0 con 0x7f7f84075770 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.128+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8406a230 con 0x7f7f84075770 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.130+0000 7f7f7affd700 1 -- 172.21.15.77:0/520335017 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7f80018020 con 0x7f7f84075770 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.131+0000 7f7f7affd700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7f700b2910 0x7f7f700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.131+0000 7f7f89469700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7f700b2910 0x7f7f700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.131+0000 7f7f7affd700 1 -- 172.21.15.77:0/520335017 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f7f800e2110 con 0x7f7f84075770 2024-05-27T08:36:46.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.131+0000 7f7f89469700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7f700b2910 0x7f7f700b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f84111b00 tx=0x7f7f7400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:46.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.135+0000 7f7f7affd700 1 -- 172.21.15.77:0/520335017 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f8001f020 con 0x7f7f84075770 2024-05-27T08:36:46.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.344+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7f8419d0c0 con 0x7f7f84075770 2024-05-27T08:36:46.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.345+0000 7f7f7affd700 1 -- 172.21.15.77:0/520335017 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7f8419d0c0 con 0x7f7f84075770 2024-05-27T08:36:46.347 INFO:teuthology.orchestra.run.smithi077.stdout:115964117067 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7f700b2910 msgr2=0x7f7f700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7f700b2910 0x7f7f700b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7f84111b00 tx=0x7f7f7400a380 comp rx=0 tx=0).stop 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84075770 msgr2=0x7f7f8419bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84075770 0x7f7f8419bdb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000b530 tx=0x7f7f8000b840 comp rx=0 tx=0).stop 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 shutdown_connections 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f7f700b2910 0x7f7f700b4dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f8410ff40 0x7f7f841a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f84075770 0x7f7f8419bdb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 --2- 172.21.15.77:0/520335017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7f84076090 0x7f7f8419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 >> 172.21.15.77:0/520335017 conn(0x7f7f8407a6a0 msgr2=0x7f7f84100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 shutdown_connections 2024-05-27T08:36:46.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:46.347+0000 7f7f8bece700 1 -- 172.21.15.77:0/520335017 wait complete. 2024-05-27T08:36:46.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:46 smithi077 ceph-mon[32007]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:46.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:46 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/520335017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T08:36:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:46 smithi179 ceph-mon[36481]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:46.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:46 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/520335017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T08:36:46.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:46 smithi092 ceph-mon[36049]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:46.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:46 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/520335017' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-27T08:36:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.200+0000 7ff77c3bd700 1 -- 172.21.15.77:0/874998882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 msgr2=0x7ff774103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.200+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/874998882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 0x7ff774103bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7700099c0 tx=0x7ff770009cd0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.200+0000 7ff77c3bd700 1 -- 172.21.15.77:0/874998882 shutdown_connections 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.200+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/874998882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7741088f0 0x7ff774110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.200+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/874998882 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 0x7ff774103bd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/874998882 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7741041a0 0x7ff7741081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 -- 172.21.15.77:0/874998882 >> 172.21.15.77:0/874998882 conn(0x7ff77407a6a0 msgr2=0x7ff77407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 -- 172.21.15.77:0/874998882 shutdown_connections 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 -- 172.21.15.77:0/874998882 wait complete. 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 Processor -- start 2024-05-27T08:36:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 -- start start 2024-05-27T08:36:47.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 0x7ff77419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:47.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7741041a0 0x7ff77419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:47.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7741088f0 0x7ff774199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:47.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff774113740 con 0x7ff7741037f0 2024-05-27T08:36:47.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741135c0 con 0x7ff7741088f0 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.201+0000 7ff77c3bd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff7741138c0 con 0x7ff7741041a0 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 0x7ff77419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 0x7ff77419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45166/0 (socket says 172.21.15.77:45166) 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 -- 172.21.15.77:0/3479507480 learned_addr learned my addr 172.21.15.77:0/3479507480 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff779958700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7741041a0 0x7ff77419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 -- 172.21.15.77:0/3479507480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7741041a0 msgr2=0x7ff77419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7741041a0 0x7ff77419fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a95a700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7741088f0 0x7ff774199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 -- 172.21.15.77:0/3479507480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7741088f0 msgr2=0x7ff774199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7741088f0 0x7ff774199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 -- 172.21.15.77:0/3479507480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77419a1a0 con 0x7ff7741037f0 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a95a700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7741088f0 0x7ff774199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:36:47.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77a159700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 0x7ff77419f830 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff770009fd0 tx=0x7ff770005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:47.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff76b7fe700 1 -- 172.21.15.77:0/3479507480 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770004830 con 0x7ff7741037f0 2024-05-27T08:36:47.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff76b7fe700 1 -- 172.21.15.77:0/3479507480 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff770005020 con 0x7ff7741037f0 2024-05-27T08:36:47.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77419a430 con 0x7ff7741037f0 2024-05-27T08:36:47.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff76b7fe700 1 -- 172.21.15.77:0/3479507480 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770003ef0 con 0x7ff7741037f0 2024-05-27T08:36:47.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.202+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7741a6700 con 0x7ff7741037f0 2024-05-27T08:36:47.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.203+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77406a230 con 0x7ff7741037f0 2024-05-27T08:36:47.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.205+0000 7ff76b7fe700 1 -- 172.21.15.77:0/3479507480 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff770029060 con 0x7ff7741037f0 2024-05-27T08:36:47.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.206+0000 7ff76b7fe700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff7600b2910 0x7ff7600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:47.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.206+0000 7ff779958700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff7600b2910 0x7ff7600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:47.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.206+0000 7ff76b7fe700 1 -- 172.21.15.77:0/3479507480 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff7700e30f0 con 0x7ff7741037f0 2024-05-27T08:36:47.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.206+0000 7ff779958700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff7600b2910 0x7ff7600b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff764005950 tx=0x7ff7640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:47.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.209+0000 7ff76b7fe700 1 -- 172.21.15.77:0/3479507480 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff770013070 con 0x7ff7741037f0 2024-05-27T08:36:47.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.425+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff77402e560 con 0x7ff7741037f0 2024-05-27T08:36:47.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.426+0000 7ff76b7fe700 1 -- 172.21.15.77:0/3479507480 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff7700aad00 con 0x7ff7741037f0 2024-05-27T08:36:47.428 INFO:teuthology.orchestra.run.smithi077.stdout:197568495676 2024-05-27T08:36:47.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff7600b2910 msgr2=0x7ff7600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:47.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff7600b2910 0x7ff7600b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff764005950 tx=0x7ff7640058e0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 msgr2=0x7ff77419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 0x7ff77419f830 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff770009fd0 tx=0x7ff770005f10 comp rx=0 tx=0).stop 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 shutdown_connections 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff7741088f0 0x7ff774199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7741037f0 0x7ff77419f830 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff7600b2910 0x7ff7600b4dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 --2- 172.21.15.77:0/3479507480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7741041a0 0x7ff77419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 >> 172.21.15.77:0/3479507480 conn(0x7ff77407a6a0 msgr2=0x7ff774101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 shutdown_connections 2024-05-27T08:36:47.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.428+0000 7ff77c3bd700 1 -- 172.21.15.77:0/3479507480 wait complete. 2024-05-27T08:36:47.724 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:47 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3479507480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T08:36:47.755 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117061 got 115964117067 for osd.3 2024-05-27T08:36:47.755 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:47 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3479507480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T08:36:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3424830841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf68102a80 msgr2=0x7fdf68102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3424830841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf68102a80 0x7fdf68102e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdf580099c0 tx=0x7fdf58009cd0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3424830841 shutdown_connections 2024-05-27T08:36:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3424830841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf681077d0 0x7fdf6810f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3424830841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf68102a80 0x7fdf68102e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3424830841 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf68103430 0x7fdf681070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3424830841 >> 172.21.15.77:0/3424830841 conn(0x7fdf680793f0 msgr2=0x7fdf68079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3424830841 shutdown_connections 2024-05-27T08:36:47.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.933+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3424830841 wait complete. 2024-05-27T08:36:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.934+0000 7fdf6dcfd700 1 Processor -- start 2024-05-27T08:36:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.934+0000 7fdf6dcfd700 1 -- start start 2024-05-27T08:36:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.934+0000 7fdf6dcfd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf68102a80 0x7fdf6819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:47.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.934+0000 7fdf6dcfd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf68103430 0x7fdf6819b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.934+0000 7fdf6dcfd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf681077d0 0x7fdf681a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.934+0000 7fdf6dcfd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdf681126b0 con 0x7fdf681077d0 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.934+0000 7fdf6dcfd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68112530 con 0x7fdf68102a80 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.934+0000 7fdf6dcfd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf68112830 con 0x7fdf68103430 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf681077d0 0x7fdf681a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf681077d0 0x7fdf681a5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45180/0 (socket says 172.21.15.77:45180) 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 -- 172.21.15.77:0/3743518807 learned_addr learned my addr 172.21.15.77:0/3743518807 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf677fe700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf68102a80 0x7fdf6819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf66ffd700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf68103430 0x7fdf6819b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 -- 172.21.15.77:0/3743518807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf68103430 msgr2=0x7fdf6819b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf68103430 0x7fdf6819b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 -- 172.21.15.77:0/3743518807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf68102a80 msgr2=0x7fdf6819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf68102a80 0x7fdf6819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 -- 172.21.15.77:0/3743518807 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf681a60d0 con 0x7fdf681077d0 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf67fff700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf681077d0 0x7fdf681a5b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000b970 tx=0x7fdf6000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf64ff9700 1 -- 172.21.15.77:0/3743518807 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000d860 con 0x7fdf681077d0 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf681a6300 con 0x7fdf681077d0 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.935+0000 7fdf64ff9700 1 -- 172.21.15.77:0/3743518807 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf6000e430 con 0x7fdf681077d0 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.936+0000 7fdf64ff9700 1 -- 172.21.15.77:0/3743518807 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf60017660 con 0x7fdf681077d0 2024-05-27T08:36:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.936+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf681a67c0 con 0x7fdf681077d0 2024-05-27T08:36:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.937+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf68103b80 con 0x7fdf681077d0 2024-05-27T08:36:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.938+0000 7fdf64ff9700 1 -- 172.21.15.77:0/3743518807 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdf6000da00 con 0x7fdf681077d0 2024-05-27T08:36:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.939+0000 7fdf64ff9700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdf500b2910 0x7fdf500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.939+0000 7fdf677fe700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdf500b2910 0x7fdf500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.939+0000 7fdf64ff9700 1 -- 172.21.15.77:0/3743518807 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fdf600e1fc0 con 0x7fdf681077d0 2024-05-27T08:36:47.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.939+0000 7fdf677fe700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdf500b2910 0x7fdf500b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009990 tx=0x7fdf58009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:47.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:47.947+0000 7fdf64ff9700 1 -- 172.21.15.77:0/3743518807 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf60013030 con 0x7fdf681077d0 2024-05-27T08:36:47.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:47 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3479507480' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-27T08:36:48.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.167+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fdf6802e560 con 0x7fdf681077d0 2024-05-27T08:36:48.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.167+0000 7fdf64ff9700 1 -- 172.21.15.77:0/3743518807 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf600a9b80 con 0x7fdf681077d0 2024-05-27T08:36:48.170 INFO:teuthology.orchestra.run.smithi077.stdout:34359738458 2024-05-27T08:36:48.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdf500b2910 msgr2=0x7fdf500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:48.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdf500b2910 0x7fdf500b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009990 tx=0x7fdf58009650 comp rx=0 tx=0).stop 2024-05-27T08:36:48.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf681077d0 msgr2=0x7fdf681a5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:48.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf681077d0 0x7fdf681a5b90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000b970 tx=0x7fdf6000bc80 comp rx=0 tx=0).stop 2024-05-27T08:36:48.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 shutdown_connections 2024-05-27T08:36:48.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdf681077d0 0x7fdf681a5b90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdf68102a80 0x7fdf6819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fdf500b2910 0x7fdf500b4dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 --2- 172.21.15.77:0/3743518807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf68103430 0x7fdf6819b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.170+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 >> 172.21.15.77:0/3743518807 conn(0x7fdf680793f0 msgr2=0x7fdf68100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:48.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.171+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 shutdown_connections 2024-05-27T08:36:48.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.171+0000 7fdf6dcfd700 1 -- 172.21.15.77:0/3743518807 wait complete. 2024-05-27T08:36:48.369 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.366+0000 7fecc2965700 1 -- 172.21.15.77:0/2729869965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc06bbe0 msgr2=0x7fecbc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.366+0000 7fecc2965700 1 --2- 172.21.15.77:0/2729869965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc06bbe0 0x7fecbc1103d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fecb400b0a0 tx=0x7fecb400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 -- 172.21.15.77:0/2729869965 shutdown_connections 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 --2- 172.21.15.77:0/2729869965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc06bbe0 0x7fecbc1103d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 --2- 172.21.15.77:0/2729869965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc1095b0 0x7fecbc109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 --2- 172.21.15.77:0/2729869965 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecbc06b220 0x7fecbc06b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 -- 172.21.15.77:0/2729869965 >> 172.21.15.77:0/2729869965 conn(0x7fecbc0793f0 msgr2=0x7fecbc079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 -- 172.21.15.77:0/2729869965 shutdown_connections 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 -- 172.21.15.77:0/2729869965 wait complete. 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 Processor -- start 2024-05-27T08:36:48.370 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.367+0000 7fecc2965700 1 -- start start 2024-05-27T08:36:48.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc2965700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecbc06b220 0x7fecbc102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:48.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc2965700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc06bbe0 0x7fecbc102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:48.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc2965700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc1095b0 0x7fecbc102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:48.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc2965700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1136c0 con 0x7fecbc1095b0 2024-05-27T08:36:48.372 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc2965700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc113540 con 0x7fecbc06b220 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc2965700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc113840 con 0x7fecbc06bbe0 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecbb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc06bbe0 0x7fecbc102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecbb7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc06bbe0 0x7fecbc102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.77:40226/0 (socket says 172.21.15.77:40226) 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc0f02700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc1095b0 0x7fecbc102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecbb7fe700 1 -- 172.21.15.77:0/4076602267 learned_addr learned my addr 172.21.15.77:0/4076602267 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecbbfff700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecbc06b220 0x7fecbc102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc0f02700 1 -- 172.21.15.77:0/4076602267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc06bbe0 msgr2=0x7fecbc102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc0f02700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc06bbe0 0x7fecbc102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc0f02700 1 -- 172.21.15.77:0/4076602267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecbc06b220 msgr2=0x7fecbc102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc0f02700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecbc06b220 0x7fecbc102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.368+0000 7fecc0f02700 1 -- 172.21.15.77:0/4076602267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc1a8180 con 0x7fecbc1095b0 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.369+0000 7fecc0f02700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc1095b0 0x7fecbc102f90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fecb400d010 tx=0x7fecb4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:48.373 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.369+0000 7fecb97fa700 1 -- 172.21.15.77:0/4076602267 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4021070 con 0x7fecbc1095b0 2024-05-27T08:36:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.369+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc1a8410 con 0x7fecbc1095b0 2024-05-27T08:36:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.369+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc1a8910 con 0x7fecbc1095b0 2024-05-27T08:36:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.369+0000 7fecb97fa700 1 -- 172.21.15.77:0/4076602267 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb4007e30 con 0x7fecbc1095b0 2024-05-27T08:36:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.369+0000 7fecb97fa700 1 -- 172.21.15.77:0/4076602267 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40045a0 con 0x7fecbc1095b0 2024-05-27T08:36:48.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.370+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca8005330 con 0x7fecbc1095b0 2024-05-27T08:36:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.373+0000 7fecb97fa700 1 -- 172.21.15.77:0/4076602267 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fecb4004740 con 0x7fecbc1095b0 2024-05-27T08:36:48.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.373+0000 7fecb97fa700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feca40b2990 0x7feca40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.373+0000 7fecbbfff700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feca40b2990 0x7feca40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.373+0000 7fecb97fa700 1 -- 172.21.15.77:0/4076602267 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fecb400f020 con 0x7fecbc1095b0 2024-05-27T08:36:48.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.374+0000 7fecbbfff700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feca40b2990 0x7feca40b4e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fecac005950 tx=0x7fecac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:48.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.376+0000 7fecb97fa700 1 -- 172.21.15.77:0/4076602267 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fecb402a020 con 0x7fecbc1095b0 2024-05-27T08:36:48.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.608+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7feca80058f0 con 0x7fecbc1095b0 2024-05-27T08:36:48.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.608+0000 7fecb97fa700 1 -- 172.21.15.77:0/4076602267 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fecb40ab5d0 con 0x7fecbc1095b0 2024-05-27T08:36:48.611 INFO:teuthology.orchestra.run.smithi077.stdout:171798691906 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feca40b2990 msgr2=0x7feca40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feca40b2990 0x7feca40b4e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fecac005950 tx=0x7fecac00b410 comp rx=0 tx=0).stop 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc1095b0 msgr2=0x7fecbc102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc1095b0 0x7fecbc102f90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fecb400d010 tx=0x7fecb4004d00 comp rx=0 tx=0).stop 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 shutdown_connections 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fecbc1095b0 0x7fecbc102f90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fecbc06b220 0x7fecbc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7feca40b2990 0x7feca40b4e50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 --2- 172.21.15.77:0/4076602267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc06bbe0 0x7fecbc102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 >> 172.21.15.77:0/4076602267 conn(0x7fecbc0793f0 msgr2=0x7fecbc101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:48.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 shutdown_connections 2024-05-27T08:36:48.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.611+0000 7fecc2965700 1 -- 172.21.15.77:0/4076602267 wait complete. 2024-05-27T08:36:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3743518807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T08:36:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1589213853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T08:36:48.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:36:48.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:36:48.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:48.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:48 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/4076602267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T08:36:49.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:49.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:49.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:49.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3743518807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T08:36:49.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1589213853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T08:36:49.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:36:49.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:36:49.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:49.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:48 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/4076602267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T08:36:49.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:49.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:49.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:49.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3743518807' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-27T08:36:49.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1589213853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-27T08:36:49.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:36:49.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:36:49.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:36:49.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:48 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/4076602267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-27T08:36:50.308 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495672 got 197568495676 for osd.6 2024-05-27T08:36:50.309 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 -- 172.21.15.77:0/3339830951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 msgr2=0x7f5698103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:50.374 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 --2- 172.21.15.77:0/3339830951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 0x7f5698103c30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56880099c0 tx=0x7f5688009cd0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 -- 172.21.15.77:0/3339830951 shutdown_connections 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 --2- 172.21.15.77:0/3339830951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698108920 0x7f56981104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 --2- 172.21.15.77:0/3339830951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 0x7f5698103c30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 --2- 172.21.15.77:0/3339830951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5698104200 0x7f56981081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 -- 172.21.15.77:0/3339830951 >> 172.21.15.77:0/3339830951 conn(0x7f569807a5d0 msgr2=0x7f569807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 -- 172.21.15.77:0/3339830951 shutdown_connections 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.154+0000 7f569d96c700 1 -- 172.21.15.77:0/3339830951 wait complete. 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.155+0000 7f569d96c700 1 Processor -- start 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.155+0000 7f569d96c700 1 -- start start 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.155+0000 7f569d96c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 0x7f569819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.155+0000 7f569d96c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698104200 0x7f569819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f569d96c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5698108920 0x7f5698199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:50.375 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f569d96c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5698113710 con 0x7f5698103850 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f569d96c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5698113590 con 0x7f5698108920 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f569d96c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5698113890 con 0x7f5698104200 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 0x7f569819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 0x7f569819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45190/0 (socket says 172.21.15.77:45190) 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 -- 172.21.15.77:0/1589213853 learned_addr learned my addr 172.21.15.77:0/1589213853 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f56967fc700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698104200 0x7f569819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f56977fe700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5698108920 0x7f5698199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 -- 172.21.15.77:0/1589213853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698104200 msgr2=0x7f569819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698104200 0x7f569819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 -- 172.21.15.77:0/1589213853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5698108920 msgr2=0x7f5698199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5698108920 0x7f5698199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 -- 172.21.15.77:0/1589213853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569819a0f0 con 0x7f5698103850 2024-05-27T08:36:50.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f5696ffd700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 0x7f569819f7d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5688006010 tx=0x7f56880045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f569c96a700 1 -- 172.21.15.77:0/1589213853 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688026070 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f569c96a700 1 -- 172.21.15.77:0/1589213853 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f568800f460 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569819a380 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.156+0000 7f569c96a700 1 -- 172.21.15.77:0/1589213853 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688009e60 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.157+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56981a66a0 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.157+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5698104cb0 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.159+0000 7f569c96a700 1 -- 172.21.15.77:0/1589213853 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5688029070 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.159+0000 7f569c96a700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f56800b2910 0x7f56800b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.160+0000 7f56967fc700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f56800b2910 0x7f56800b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.160+0000 7f569c96a700 1 -- 172.21.15.77:0/1589213853 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f56880e2ec0 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.160+0000 7f56967fc700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f56800b2910 0x7f56800b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5698103140 tx=0x7f568c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.164+0000 7f569c96a700 1 -- 172.21.15.77:0/1589213853 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5688030080 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.419+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f569802e560 con 0x7f5698103850 2024-05-27T08:36:50.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.419+0000 7f569c96a700 1 -- 172.21.15.77:0/1589213853 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f5688013070 con 0x7f5698103850 2024-05-27T08:36:50.378 INFO:teuthology.orchestra.run.smithi077.stdout:296352743465 2024-05-27T08:36:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f56800b2910 msgr2=0x7f56800b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f56800b2910 0x7f56800b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5698103140 tx=0x7f568c00b410 comp rx=0 tx=0).stop 2024-05-27T08:36:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 msgr2=0x7f569819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 0x7f569819f7d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5688006010 tx=0x7f56880045e0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 shutdown_connections 2024-05-27T08:36:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5698108920 0x7f5698199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5698103850 0x7f569819f7d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f56800b2910 0x7f56800b4dd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 --2- 172.21.15.77:0/1589213853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5698104200 0x7f569819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 >> 172.21.15.77:0/1589213853 conn(0x7f569807a5d0 msgr2=0x7f56981017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 shutdown_connections 2024-05-27T08:36:50.379 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:48.422+0000 7f569d96c700 1 -- 172.21.15.77:0/1589213853 wait complete. 2024-05-27T08:36:50.962 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:50 smithi077 ceph-mon[32007]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:50.988 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743458 got 296352743465 for osd.10 2024-05-27T08:36:50.988 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:51.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:50 smithi179 ceph-mon[36481]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:51.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:50 smithi092 ceph-mon[36049]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:36:51.602 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135918 got 244813135922 for osd.8 2024-05-27T08:36:51.602 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:51.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.794+0000 7f161ba34700 1 -- 172.21.15.77:0/268373044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614108620 msgr2=0x7f1614108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:51.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.794+0000 7f161ba34700 1 --2- 172.21.15.77:0/268373044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614108620 0x7f1614108a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1610009990 tx=0x7f1610009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:51.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.794+0000 7f161ba34700 1 -- 172.21.15.77:0/268373044 shutdown_connections 2024-05-27T08:36:51.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.794+0000 7f161ba34700 1 --2- 172.21.15.77:0/268373044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16141028c0 0x7f1614110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.795+0000 7f161ba34700 1 --2- 172.21.15.77:0/268373044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614108620 0x7f1614108a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.795+0000 7f161ba34700 1 --2- 172.21.15.77:0/268373044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1614101f00 0x7f1614102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.795+0000 7f161ba34700 1 -- 172.21.15.77:0/268373044 >> 172.21.15.77:0/268373044 conn(0x7f1614077780 msgr2=0x7f1614077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.795+0000 7f161ba34700 1 -- 172.21.15.77:0/268373044 shutdown_connections 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.795+0000 7f161ba34700 1 -- 172.21.15.77:0/268373044 wait complete. 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.795+0000 7f161ba34700 1 Processor -- start 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.795+0000 7f161ba34700 1 -- start start 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f161ba34700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614101f00 0x7f161419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f161ba34700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16141028c0 0x7f161419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f161ba34700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1614108620 0x7f1614199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f161ba34700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f16141137a0 con 0x7f1614101f00 2024-05-27T08:36:51.800 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f161ba34700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1614113620 con 0x7f1614108620 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f161ba34700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1614113920 con 0x7f16141028c0 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614101f00 0x7f161419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614101f00 0x7f161419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56824/0 (socket says 172.21.15.77:56824) 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 -- 172.21.15.77:0/1341053089 learned_addr learned my addr 172.21.15.77:0/1341053089 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f1618fcf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16141028c0 0x7f161419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f1619fd1700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1614108620 0x7f1614199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 -- 172.21.15.77:0/1341053089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16141028c0 msgr2=0x7f161419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16141028c0 0x7f161419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 -- 172.21.15.77:0/1341053089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1614108620 msgr2=0x7f1614199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1614108620 0x7f1614199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 -- 172.21.15.77:0/1341053089 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161419a260 con 0x7f1614101f00 2024-05-27T08:36:51.801 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.796+0000 7f16197d0700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614101f00 0x7f161419f8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1610009f50 tx=0x7f1610009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:51.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.798+0000 7f160a7fc700 1 -- 172.21.15.77:0/1341053089 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610027070 con 0x7f1614101f00 2024-05-27T08:36:51.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.798+0000 7f160a7fc700 1 -- 172.21.15.77:0/1341053089 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1610003fb0 con 0x7f1614101f00 2024-05-27T08:36:51.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.798+0000 7f160a7fc700 1 -- 172.21.15.77:0/1341053089 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610022d00 con 0x7f1614101f00 2024-05-27T08:36:51.802 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.798+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161419a4f0 con 0x7f1614101f00 2024-05-27T08:36:51.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.798+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16141a67c0 con 0x7f1614101f00 2024-05-27T08:36:51.803 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.799+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161406a230 con 0x7f1614101f00 2024-05-27T08:36:51.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.802+0000 7f160a7fc700 1 -- 172.21.15.77:0/1341053089 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f161002a070 con 0x7f1614101f00 2024-05-27T08:36:51.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.802+0000 7f160a7fc700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16000b2990 0x7f16000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:51.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.802+0000 7f1618fcf700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16000b2990 0x7f16000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:51.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.802+0000 7f160a7fc700 1 -- 172.21.15.77:0/1341053089 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f16100e3a30 con 0x7f1614101f00 2024-05-27T08:36:51.804 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.802+0000 7f1618fcf700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16000b2990 0x7f16000b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1614077dd0 tx=0x7f160400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:51.807 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:51.805+0000 7f160a7fc700 1 -- 172.21.15.77:0/1341053089 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1610031070 con 0x7f1614101f00 2024-05-27T08:36:52.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.015+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f161402e560 con 0x7f1614101f00 2024-05-27T08:36:52.017 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.015+0000 7f160a7fc700 1 -- 172.21.15.77:0/1341053089 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1610013070 con 0x7f1614101f00 2024-05-27T08:36:52.018 INFO:teuthology.orchestra.run.smithi077.stdout:219043332151 2024-05-27T08:36:52.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16000b2990 msgr2=0x7f16000b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:52.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16000b2990 0x7f16000b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1614077dd0 tx=0x7f160400a300 comp rx=0 tx=0).stop 2024-05-27T08:36:52.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614101f00 msgr2=0x7f161419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:52.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614101f00 0x7f161419f8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1610009f50 tx=0x7f1610009f80 comp rx=0 tx=0).stop 2024-05-27T08:36:52.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 shutdown_connections 2024-05-27T08:36:52.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1614108620 0x7f1614199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1614101f00 0x7f161419f8f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f16000b2990 0x7f16000b4e50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 --2- 172.21.15.77:0/1341053089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16141028c0 0x7f161419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 >> 172.21.15.77:0/1341053089 conn(0x7f1614077780 msgr2=0x7f1614100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:52.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 shutdown_connections 2024-05-27T08:36:52.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.018+0000 7f161ba34700 1 -- 172.21.15.77:0/1341053089 wait complete. 2024-05-27T08:36:52.164 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691902 got 171798691906 for osd.5 2024-05-27T08:36:52.164 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 -- 172.21.15.77:0/1782235894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f30103600 msgr2=0x7f8f3010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:52.334 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 --2- 172.21.15.77:0/1782235894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f30103600 0x7f8f3010f2e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b0a0 tx=0x7f8f2800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 -- 172.21.15.77:0/1782235894 shutdown_connections 2024-05-27T08:36:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 --2- 172.21.15.77:0/1782235894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f30103600 0x7f8f3010f2e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 --2- 172.21.15.77:0/1782235894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f301084c0 0x7f8f301088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 --2- 172.21.15.77:0/1782235894 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f30102c40 0x7f8f301030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 -- 172.21.15.77:0/1782235894 >> 172.21.15.77:0/1782235894 conn(0x7f8f30078ab0 msgr2=0x7f8f300ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 -- 172.21.15.77:0/1782235894 shutdown_connections 2024-05-27T08:36:52.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.333+0000 7f8f3622c700 1 -- 172.21.15.77:0/1782235894 wait complete. 2024-05-27T08:36:52.336 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f3622c700 1 Processor -- start 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f3622c700 1 -- start start 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f3622c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f30102c40 0x7f8f3019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f3622c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f30103600 0x7f8f3019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f3622c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f301084c0 0x7f8f301a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f3622c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30112670 con 0x7f8f301084c0 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f3622c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301124f0 con 0x7f8f30102c40 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f3622c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301127f0 con 0x7f8f30103600 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f2ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f301084c0 0x7f8f301a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f2ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f301084c0 0x7f8f301a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56842/0 (socket says 172.21.15.77:56842) 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f2ffff700 1 -- 172.21.15.77:0/815595295 learned_addr learned my addr 172.21.15.77:0/815595295 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f2f7fe700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f30102c40 0x7f8f3019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.334+0000 7f8f2effd700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f30103600 0x7f8f3019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:52.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2ffff700 1 -- 172.21.15.77:0/815595295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f30103600 msgr2=0x7f8f3019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2ffff700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f30103600 0x7f8f3019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2ffff700 1 -- 172.21.15.77:0/815595295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f30102c40 msgr2=0x7f8f3019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2ffff700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f30102c40 0x7f8f3019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2ffff700 1 -- 172.21.15.77:0/815595295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f301a6030 con 0x7f8f301084c0 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2ffff700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f301084c0 0x7f8f301a5af0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b830 tx=0x7f8f28004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2cff9700 1 -- 172.21.15.77:0/815595295 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28021070 con 0x7f8f301084c0 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f301a6200 con 0x7f8f301084c0 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2cff9700 1 -- 172.21.15.77:0/815595295 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f28007e30 con 0x7f8f301084c0 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f2cff9700 1 -- 172.21.15.77:0/815595295 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280047d0 con 0x7f8f301084c0 2024-05-27T08:36:52.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.335+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f301a66b0 con 0x7f8f301084c0 2024-05-27T08:36:52.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.336+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f30103c50 con 0x7f8f301084c0 2024-05-27T08:36:52.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.337+0000 7f8f2cff9700 1 -- 172.21.15.77:0/815595295 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8f28007980 con 0x7f8f301084c0 2024-05-27T08:36:52.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.338+0000 7f8f2cff9700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8f180b2910 0x7f8f180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:52.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.338+0000 7f8f2f7fe700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8f180b2910 0x7f8f180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:52.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.338+0000 7f8f2cff9700 1 -- 172.21.15.77:0/815595295 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f8f2800f020 con 0x7f8f301084c0 2024-05-27T08:36:52.340 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.338+0000 7f8f2f7fe700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8f180b2910 0x7f8f180b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f3006ae90 tx=0x7f8f20009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:52.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.341+0000 7f8f2cff9700 1 -- 172.21.15.77:0/815595295 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f2802a020 con 0x7f8f301084c0 2024-05-27T08:36:52.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.554+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f8f3002e560 con 0x7f8f301084c0 2024-05-27T08:36:52.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.554+0000 7f8f2cff9700 1 -- 172.21.15.77:0/815595295 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8f280ab9a0 con 0x7f8f301084c0 2024-05-27T08:36:52.555 INFO:teuthology.orchestra.run.smithi077.stdout:270582939694 2024-05-27T08:36:52.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8f180b2910 msgr2=0x7f8f180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:52.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8f180b2910 0x7f8f180b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f3006ae90 tx=0x7f8f20009190 comp rx=0 tx=0).stop 2024-05-27T08:36:52.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f301084c0 msgr2=0x7f8f301a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:52.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f301084c0 0x7f8f301a5af0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b830 tx=0x7f8f28004ce0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 shutdown_connections 2024-05-27T08:36:52.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f301084c0 0x7f8f301a5af0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8f30102c40 0x7f8f3019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f8f180b2910 0x7f8f180b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 --2- 172.21.15.77:0/815595295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8f30103600 0x7f8f3019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:52.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.556+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 >> 172.21.15.77:0/815595295 conn(0x7f8f30078ab0 msgr2=0x7f8f30100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:52.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.557+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 shutdown_connections 2024-05-27T08:36:52.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:52.557+0000 7f8f3622c700 1 -- 172.21.15.77:0/815595295 wait complete. 2024-05-27T08:36:52.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:52 smithi077 ceph-mon[32007]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:52.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:52 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1341053089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T08:36:52.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:52 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/815595295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T08:36:53.031 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332144 got 219043332151 for osd.7 2024-05-27T08:36:53.031 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:53.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:52 smithi179 ceph-mon[36481]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:53.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:52 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1341053089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T08:36:53.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:52 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/815595295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T08:36:53.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:52 smithi092 ceph-mon[36049]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:36:53.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:52 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1341053089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-27T08:36:53.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:52 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/815595295' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-27T08:36:53.459 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738458 for osd.0 2024-05-27T08:36:53.459 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:54.075 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939689 got 270582939694 for osd.9 2024-05-27T08:36:54.076 DEBUG:teuthology.parallel:result is None 2024-05-27T08:36:54.076 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T08:36:54.076 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T08:36:54.320 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:55.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:54 smithi179 ceph-mon[36481]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:55.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:54 smithi077 ceph-mon[32007]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:55.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:54 smithi092 ceph-mon[36049]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:36:56.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.041+0000 7f270952e700 1 -- 172.21.15.77:0/1376333849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f270406bbe0 msgr2=0x7f27041103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:56.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.041+0000 7f270952e700 1 --2- 172.21.15.77:0/1376333849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f270406bbe0 0x7f27041103d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00b0a0 tx=0x7f26fc00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.041+0000 7f270952e700 1 -- 172.21.15.77:0/1376333849 shutdown_connections 2024-05-27T08:36:56.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.041+0000 7f270952e700 1 --2- 172.21.15.77:0/1376333849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f270406bbe0 0x7f27041103d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.045 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.041+0000 7f270952e700 1 --2- 172.21.15.77:0/1376333849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27041095b0 0x7f2704109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.041+0000 7f270952e700 1 --2- 172.21.15.77:0/1376333849 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270406b220 0x7f270406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.041+0000 7f270952e700 1 -- 172.21.15.77:0/1376333849 >> 172.21.15.77:0/1376333849 conn(0x7f27040793f0 msgr2=0x7f2704079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.042+0000 7f270952e700 1 -- 172.21.15.77:0/1376333849 shutdown_connections 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.042+0000 7f270952e700 1 -- 172.21.15.77:0/1376333849 wait complete. 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.042+0000 7f270952e700 1 Processor -- start 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.042+0000 7f270952e700 1 -- start start 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f270952e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270406b220 0x7f270419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f270952e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f270406bbe0 0x7f270419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f270952e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27041095b0 0x7f27041998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f270952e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f27041136c0 con 0x7f27041095b0 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f270952e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2704113540 con 0x7f270406b220 2024-05-27T08:36:56.046 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f270952e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2704113840 con 0x7f270406bbe0 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27041095b0 0x7f27041998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27041095b0 0x7f27041998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56860/0 (socket says 172.21.15.77:56860) 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 -- 172.21.15.77:0/1906843669 learned_addr learned my addr 172.21.15.77:0/1906843669 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27027fc700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f270406bbe0 0x7f270419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f2702ffd700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270406b220 0x7f270419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 -- 172.21.15.77:0/1906843669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f270406bbe0 msgr2=0x7f270419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f270406bbe0 0x7f270419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 -- 172.21.15.77:0/1906843669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270406b220 msgr2=0x7f270419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270406b220 0x7f270419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 -- 172.21.15.77:0/1906843669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270419a000 con 0x7f27041095b0 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f27037fe700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27041095b0 0x7f27041998a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0062a0 tx=0x7f26fc004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:56.047 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f26ebfff700 1 -- 172.21.15.77:0/1906843669 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc021070 con 0x7f27041095b0 2024-05-27T08:36:56.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270419a290 con 0x7f27041095b0 2024-05-27T08:36:56.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f26ebfff700 1 -- 172.21.15.77:0/1906843669 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26fc003c00 con 0x7f27041095b0 2024-05-27T08:36:56.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f26ebfff700 1 -- 172.21.15.77:0/1906843669 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004880 con 0x7f27041095b0 2024-05-27T08:36:56.048 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.043+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27041a65b0 con 0x7f27041095b0 2024-05-27T08:36:56.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.044+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270406a230 con 0x7f27041095b0 2024-05-27T08:36:56.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.046+0000 7f26ebfff700 1 -- 172.21.15.77:0/1906843669 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f26fc004020 con 0x7f27041095b0 2024-05-27T08:36:56.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.046+0000 7f26ebfff700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f26ec0b2910 0x7f26ec0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:56.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.046+0000 7f2702ffd700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f26ec0b2910 0x7f26ec0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:56.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.046+0000 7f26ebfff700 1 -- 172.21.15.77:0/1906843669 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f26fc00f020 con 0x7f27041095b0 2024-05-27T08:36:56.049 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.047+0000 7f2702ffd700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f26ec0b2910 0x7f26ec0b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f270419a9e0 tx=0x7f26f400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:56.051 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.050+0000 7f26ebfff700 1 -- 172.21.15.77:0/1906843669 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26fc02a020 con 0x7f27041095b0 2024-05-27T08:36:56.252 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.250+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2704063980 con 0x7f26ec0b2910 2024-05-27T08:36:56.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.258+0000 7f26ebfff700 1 -- 172.21.15.77:0/1906843669 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7f2704063980 con 0x7f26ec0b2910 2024-05-27T08:36:56.264 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:36:56.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f26ec0b2910 msgr2=0x7f26ec0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:56.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f26ec0b2910 0x7f26ec0b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f270419a9e0 tx=0x7f26f400a380 comp rx=0 tx=0).stop 2024-05-27T08:36:56.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27041095b0 msgr2=0x7f27041998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:56.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27041095b0 0x7f27041998a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0062a0 tx=0x7f26fc004e20 comp rx=0 tx=0).stop 2024-05-27T08:36:56.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 shutdown_connections 2024-05-27T08:36:56.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27041095b0 0x7f27041998a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f270406b220 0x7f270419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.266 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f26ec0b2910 0x7f26ec0b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 --2- 172.21.15.77:0/1906843669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f270406bbe0 0x7f270419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:56.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 >> 172.21.15.77:0/1906843669 conn(0x7f27040793f0 msgr2=0x7f2704100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:56.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 shutdown_connections 2024-05-27T08:36:56.267 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:56.261+0000 7f270952e700 1 -- 172.21.15.77:0/1906843669 wait complete. 2024-05-27T08:36:56.267 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-05-27T08:36:56.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:56 smithi077 ceph-mon[32007]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:57.040 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-05-27T08:36:55.175081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657568,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1015,"num_read_kb":607,"num_write":371,"num_write_kb":2287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2953206,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362340,"kb_used_data":13116,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579484,"statfs":{"total":1151940427776,"available":1151569391616,"internally_reserved":0,"allocated":13430784,"data_stored":7336020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":140,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003171"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371664+0000","last_change":"2024-05-27T08:35:07.046361+0000","last_active":"2024-05-27T08:35:58.371664+0000","last_peered":"2024-05-27T08:35:58.371664+0000","last_clean":"2024-05-27T08:35:58.371664+0000","last_became_active":"2024-05-27T08:35:07.045954+0000","last_became_peered":"2024-05-27T08:35:07.045954+0000","last_unstale":"2024-05-27T08:35:58.371664+0000","last_undegraded":"2024-05-27T08:35:58.371664+0000","last_fullsized":"2024-05-27T08:35:58.371664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:03:47.458964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370683+0000","last_change":"2024-05-27T08:35:07.043725+0000","last_active":"2024-05-27T08:35:58.370683+0000","last_peered":"2024-05-27T08:35:58.370683+0000","last_clean":"2024-05-27T08:35:58.370683+0000","last_became_active":"2024-05-27T08:35:07.043447+0000","last_became_peered":"2024-05-27T08:35:07.043447+0000","last_unstale":"2024-05-27T08:35:58.370683+0000","last_undegraded":"2024-05-27T08:35:58.370683+0000","last_fullsized":"2024-05-27T08:35:58.370683+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:44.509202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370533+0000","last_change":"2024-05-27T08:35:07.048075+0000","last_active":"2024-05-27T08:35:58.370533+0000","last_peered":"2024-05-27T08:35:58.370533+0000","last_clean":"2024-05-27T08:35:58.370533+0000","last_became_active":"2024-05-27T08:35:07.047687+0000","last_became_peered":"2024-05-27T08:35:07.047687+0000","last_unstale":"2024-05-27T08:35:58.370533+0000","last_undegraded":"2024-05-27T08:35:58.370533+0000","last_fullsized":"2024-05-27T08:35:58.370533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:05:15.197231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.407512+0000","last_change":"2024-05-27T08:35:07.046809+0000","last_active":"2024-05-27T08:35:58.407512+0000","last_peered":"2024-05-27T08:35:58.407512+0000","last_clean":"2024-05-27T08:35:58.407512+0000","last_became_active":"2024-05-27T08:35:07.046453+0000","last_became_peered":"2024-05-27T08:35:07.046453+0000","last_unstale":"2024-05-27T08:35:58.407512+0000","last_undegraded":"2024-05-27T08:35:58.407512+0000","last_fullsized":"2024-05-27T08:35:58.407512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:43.209613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368023+0000","last_change":"2024-05-27T08:35:07.040242+0000","last_active":"2024-05-27T08:35:58.368023+0000","last_peered":"2024-05-27T08:35:58.368023+0000","last_clean":"2024-05-27T08:35:58.368023+0000","last_became_active":"2024-05-27T08:35:07.039860+0000","last_became_peered":"2024-05-27T08:35:07.039860+0000","last_unstale":"2024-05-27T08:35:58.368023+0000","last_undegraded":"2024-05-27T08:35:58.368023+0000","last_fullsized":"2024-05-27T08:35:58.368023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:49.212799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367633+0000","last_change":"2024-05-27T08:35:07.048176+0000","last_active":"2024-05-27T08:35:58.367633+0000","last_peered":"2024-05-27T08:35:58.367633+0000","last_clean":"2024-05-27T08:35:58.367633+0000","last_became_active":"2024-05-27T08:35:07.047860+0000","last_became_peered":"2024-05-27T08:35:07.047860+0000","last_unstale":"2024-05-27T08:35:58.367633+0000","last_undegraded":"2024-05-27T08:35:58.367633+0000","last_fullsized":"2024-05-27T08:35:58.367633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:19.459704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370920+0000","last_change":"2024-05-27T08:35:07.052285+0000","last_active":"2024-05-27T08:35:58.370920+0000","last_peered":"2024-05-27T08:35:58.370920+0000","last_clean":"2024-05-27T08:35:58.370920+0000","last_became_active":"2024-05-27T08:35:07.051970+0000","last_became_peered":"2024-05-27T08:35:07.051970+0000","last_unstale":"2024-05-27T08:35:58.370920+0000","last_undegraded":"2024-05-27T08:35:58.370920+0000","last_fullsized":"2024-05-27T08:35:58.370920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:56.543022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367632+0000","last_change":"2024-05-27T08:35:07.047527+0000","last_active":"2024-05-27T08:35:58.367632+0000","last_peered":"2024-05-27T08:35:58.367632+0000","last_clean":"2024-05-27T08:35:58.367632+0000","last_became_active":"2024-05-27T08:35:07.047204+0000","last_became_peered":"2024-05-27T08:35:07.047204+0000","last_unstale":"2024-05-27T08:35:58.367632+0000","last_undegraded":"2024-05-27T08:35:58.367632+0000","last_fullsized":"2024-05-27T08:35:58.367632+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:04:43.547847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375361+0000","last_change":"2024-05-27T08:35:07.049108+0000","last_active":"2024-05-27T08:35:58.375361+0000","last_peered":"2024-05-27T08:35:58.375361+0000","last_clean":"2024-05-27T08:35:58.375361+0000","last_became_active":"2024-05-27T08:35:07.048540+0000","last_became_peered":"2024-05-27T08:35:07.048540+0000","last_unstale":"2024-05-27T08:35:58.375361+0000","last_undegraded":"2024-05-27T08:35:58.375361+0000","last_fullsized":"2024-05-27T08:35:58.375361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:56.219362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367976+0000","last_change":"2024-05-27T08:35:07.049775+0000","last_active":"2024-05-27T08:35:58.367976+0000","last_peered":"2024-05-27T08:35:58.367976+0000","last_clean":"2024-05-27T08:35:58.367976+0000","last_became_active":"2024-05-27T08:35:07.049466+0000","last_became_peered":"2024-05-27T08:35:07.049466+0000","last_unstale":"2024-05-27T08:35:58.367976+0000","last_undegraded":"2024-05-27T08:35:58.367976+0000","last_fullsized":"2024-05-27T08:35:58.367976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:18:30.394390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371274+0000","last_change":"2024-05-27T08:35:07.050870+0000","last_active":"2024-05-27T08:35:58.371274+0000","last_peered":"2024-05-27T08:35:58.371274+0000","last_clean":"2024-05-27T08:35:58.371274+0000","last_became_active":"2024-05-27T08:35:07.050527+0000","last_became_peered":"2024-05-27T08:35:07.050527+0000","last_unstale":"2024-05-27T08:35:58.371274+0000","last_undegraded":"2024-05-27T08:35:58.371274+0000","last_fullsized":"2024-05-27T08:35:58.371274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:56:41.477053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.407517+0000","last_change":"2024-05-27T08:35:07.052955+0000","last_active":"2024-05-27T08:35:58.407517+0000","last_peered":"2024-05-27T08:35:58.407517+0000","last_clean":"2024-05-27T08:35:58.407517+0000","last_became_active":"2024-05-27T08:35:07.052500+0000","last_became_peered":"2024-05-27T08:35:07.052500+0000","last_unstale":"2024-05-27T08:35:58.407517+0000","last_undegraded":"2024-05-27T08:35:58.407517+0000","last_fullsized":"2024-05-27T08:35:58.407517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:25.093408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364731+0000","last_change":"2024-05-27T08:35:07.044866+0000","last_active":"2024-05-27T08:35:58.364731+0000","last_peered":"2024-05-27T08:35:58.364731+0000","last_clean":"2024-05-27T08:35:58.364731+0000","last_became_active":"2024-05-27T08:35:07.044554+0000","last_became_peered":"2024-05-27T08:35:07.044554+0000","last_unstale":"2024-05-27T08:35:58.364731+0000","last_undegraded":"2024-05-27T08:35:58.364731+0000","last_fullsized":"2024-05-27T08:35:58.364731+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:45.624970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370530+0000","last_change":"2024-05-27T08:35:07.043985+0000","last_active":"2024-05-27T08:35:58.370530+0000","last_peered":"2024-05-27T08:35:58.370530+0000","last_clean":"2024-05-27T08:35:58.370530+0000","last_became_active":"2024-05-27T08:35:07.043694+0000","last_became_peered":"2024-05-27T08:35:07.043694+0000","last_unstale":"2024-05-27T08:35:58.370530+0000","last_undegraded":"2024-05-27T08:35:58.370530+0000","last_fullsized":"2024-05-27T08:35:58.370530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:22:49.796670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370414+0000","last_change":"2024-05-27T08:35:07.045542+0000","last_active":"2024-05-27T08:35:58.370414+0000","last_peered":"2024-05-27T08:35:58.370414+0000","last_clean":"2024-05-27T08:35:58.370414+0000","last_became_active":"2024-05-27T08:35:07.045209+0000","last_became_peered":"2024-05-27T08:35:07.045209+0000","last_unstale":"2024-05-27T08:35:58.370414+0000","last_undegraded":"2024-05-27T08:35:58.370414+0000","last_fullsized":"2024-05-27T08:35:58.370414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:57.733089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374546+0000","last_change":"2024-05-27T08:35:07.047193+0000","last_active":"2024-05-27T08:35:58.374546+0000","last_peered":"2024-05-27T08:35:58.374546+0000","last_clean":"2024-05-27T08:35:58.374546+0000","last_became_active":"2024-05-27T08:35:07.046862+0000","last_became_peered":"2024-05-27T08:35:07.046862+0000","last_unstale":"2024-05-27T08:35:58.374546+0000","last_undegraded":"2024-05-27T08:35:58.374546+0000","last_fullsized":"2024-05-27T08:35:58.374546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:21.288642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371826+0000","last_change":"2024-05-27T08:35:07.045452+0000","last_active":"2024-05-27T08:35:58.371826+0000","last_peered":"2024-05-27T08:35:58.371826+0000","last_clean":"2024-05-27T08:35:58.371826+0000","last_became_active":"2024-05-27T08:35:07.044783+0000","last_became_peered":"2024-05-27T08:35:07.044783+0000","last_unstale":"2024-05-27T08:35:58.371826+0000","last_undegraded":"2024-05-27T08:35:58.371826+0000","last_fullsized":"2024-05-27T08:35:58.371826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:03.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368561+0000","last_change":"2024-05-27T08:35:07.042864+0000","last_active":"2024-05-27T08:35:58.368561+0000","last_peered":"2024-05-27T08:35:58.368561+0000","last_clean":"2024-05-27T08:35:58.368561+0000","last_became_active":"2024-05-27T08:35:07.042523+0000","last_became_peered":"2024-05-27T08:35:07.042523+0000","last_unstale":"2024-05-27T08:35:58.368561+0000","last_undegraded":"2024-05-27T08:35:58.368561+0000","last_fullsized":"2024-05-27T08:35:58.368561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:34.275977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.369532+0000","last_change":"2024-05-27T08:35:07.042667+0000","last_active":"2024-05-27T08:35:58.369532+0000","last_peered":"2024-05-27T08:35:58.369532+0000","last_clean":"2024-05-27T08:35:58.369532+0000","last_became_active":"2024-05-27T08:35:07.042090+0000","last_became_peered":"2024-05-27T08:35:07.042090+0000","last_unstale":"2024-05-27T08:35:58.369532+0000","last_undegraded":"2024-05-27T08:35:58.369532+0000","last_fullsized":"2024-05-27T08:35:58.369532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:28.764641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":148,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:51.444135+0000","last_change":"2024-05-27T08:35:58.391406+0000","last_active":"2024-05-27T08:36:51.444135+0000","last_peered":"2024-05-27T08:36:51.444135+0000","last_clean":"2024-05-27T08:36:51.444135+0000","last_became_active":"2024-05-27T08:35:58.376035+0000","last_became_peered":"2024-05-27T08:35:58.376035+0000","last_unstale":"2024-05-27T08:36:51.444135+0000","last_undegraded":"2024-05-27T08:36:51.444135+0000","last_fullsized":"2024-05-27T08:36:51.444135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:54.751467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.376043+0000","last_change":"2024-05-27T08:35:58.376043+0000","last_active":"2024-05-27T08:35:58.376043+0000","last_peered":"2024-05-27T08:35:58.376043+0000","last_clean":"2024-05-27T08:35:58.376043+0000","last_became_active":"2024-05-27T08:35:58.375695+0000","last_became_peered":"2024-05-27T08:35:58.375695+0000","last_unstale":"2024-05-27T08:35:58.376043+0000","last_undegraded":"2024-05-27T08:35:58.376043+0000","last_fullsized":"2024-05-27T08:35:58.376043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:30:40.875555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368162+0000","last_change":"2024-05-27T08:35:04.245013+0000","last_active":"2024-05-27T08:35:58.368162+0000","last_peered":"2024-05-27T08:35:58.368162+0000","last_clean":"2024-05-27T08:35:58.368162+0000","last_became_active":"2024-05-27T08:35:04.244629+0000","last_became_peered":"2024-05-27T08:35:04.244629+0000","last_unstale":"2024-05-27T08:35:58.368162+0000","last_undegraded":"2024-05-27T08:35:58.368162+0000","last_fullsized":"2024-05-27T08:35:58.368162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:41:43.747592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371029+0000","last_change":"2024-05-27T08:35:07.041776+0000","last_active":"2024-05-27T08:35:58.371029+0000","last_peered":"2024-05-27T08:35:58.371029+0000","last_clean":"2024-05-27T08:35:58.371029+0000","last_became_active":"2024-05-27T08:35:07.041400+0000","last_became_peered":"2024-05-27T08:35:07.041400+0000","last_unstale":"2024-05-27T08:35:58.371029+0000","last_undegraded":"2024-05-27T08:35:58.371029+0000","last_fullsized":"2024-05-27T08:35:58.371029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:10.091310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368616+0000","last_change":"2024-05-27T08:35:04.023072+0000","last_active":"2024-05-27T08:35:58.368616+0000","last_peered":"2024-05-27T08:35:58.368616+0000","last_clean":"2024-05-27T08:35:58.368616+0000","last_became_active":"2024-05-27T08:35:04.022232+0000","last_became_peered":"2024-05-27T08:35:04.022232+0000","last_unstale":"2024-05-27T08:35:58.368616+0000","last_undegraded":"2024-05-27T08:35:58.368616+0000","last_fullsized":"2024-05-27T08:35:58.368616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:49:57.240621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374796+0000","last_change":"2024-05-27T08:35:07.044727+0000","last_active":"2024-05-27T08:35:58.374796+0000","last_peered":"2024-05-27T08:35:58.374796+0000","last_clean":"2024-05-27T08:35:58.374796+0000","last_became_active":"2024-05-27T08:35:07.044352+0000","last_became_peered":"2024-05-27T08:35:07.044352+0000","last_unstale":"2024-05-27T08:35:58.374796+0000","last_undegraded":"2024-05-27T08:35:58.374796+0000","last_fullsized":"2024-05-27T08:35:58.374796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:52:43.473606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.382377+0000","last_change":"2024-05-27T08:35:58.382377+0000","last_active":"2024-05-27T08:35:58.382377+0000","last_peered":"2024-05-27T08:35:58.382377+0000","last_clean":"2024-05-27T08:35:58.382377+0000","last_became_active":"2024-05-27T08:35:58.376093+0000","last_became_peered":"2024-05-27T08:35:58.376093+0000","last_unstale":"2024-05-27T08:35:58.382377+0000","last_undegraded":"2024-05-27T08:35:58.382377+0000","last_fullsized":"2024-05-27T08:35:58.382377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:33.792015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.376269+0000","last_change":"2024-05-27T08:35:07.055562+0000","last_active":"2024-05-27T08:35:58.376269+0000","last_peered":"2024-05-27T08:35:58.376269+0000","last_clean":"2024-05-27T08:35:58.376269+0000","last_became_active":"2024-05-27T08:35:07.055209+0000","last_became_peered":"2024-05-27T08:35:07.055209+0000","last_unstale":"2024-05-27T08:35:58.376269+0000","last_undegraded":"2024-05-27T08:35:58.376269+0000","last_fullsized":"2024-05-27T08:35:58.376269+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:04.920534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:52.403622+0000","last_change":"2024-05-27T08:35:04.022778+0000","last_active":"2024-05-27T08:36:52.403622+0000","last_peered":"2024-05-27T08:36:52.403622+0000","last_clean":"2024-05-27T08:36:52.403622+0000","last_became_active":"2024-05-27T08:35:04.022311+0000","last_became_peered":"2024-05-27T08:35:04.022311+0000","last_unstale":"2024-05-27T08:36:52.403622+0000","last_undegraded":"2024-05-27T08:36:52.403622+0000","last_fullsized":"2024-05-27T08:36:52.403622+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:48.174774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367996+0000","last_change":"2024-05-27T08:35:07.040182+0000","last_active":"2024-05-27T08:35:58.367996+0000","last_peered":"2024-05-27T08:35:58.367996+0000","last_clean":"2024-05-27T08:35:58.367996+0000","last_became_active":"2024-05-27T08:35:07.039800+0000","last_became_peered":"2024-05-27T08:35:07.039800+0000","last_unstale":"2024-05-27T08:35:58.367996+0000","last_undegraded":"2024-05-27T08:35:58.367996+0000","last_fullsized":"2024-05-27T08:35:58.367996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:53:53.470284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367626+0000","last_change":"2024-05-27T08:35:04.023757+0000","last_active":"2024-05-27T08:35:58.367626+0000","last_peered":"2024-05-27T08:35:58.367626+0000","last_clean":"2024-05-27T08:35:58.367626+0000","last_became_active":"2024-05-27T08:35:04.023428+0000","last_became_peered":"2024-05-27T08:35:04.023428+0000","last_unstale":"2024-05-27T08:35:58.367626+0000","last_undegraded":"2024-05-27T08:35:58.367626+0000","last_fullsized":"2024-05-27T08:35:58.367626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:45.669006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.407697+0000","last_change":"2024-05-27T08:35:58.407697+0000","last_active":"2024-05-27T08:35:58.407697+0000","last_peered":"2024-05-27T08:35:58.407697+0000","last_clean":"2024-05-27T08:35:58.407697+0000","last_became_active":"2024-05-27T08:35:58.407379+0000","last_became_peered":"2024-05-27T08:35:58.407379+0000","last_unstale":"2024-05-27T08:35:58.407697+0000","last_undegraded":"2024-05-27T08:35:58.407697+0000","last_fullsized":"2024-05-27T08:35:58.407697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:06.534914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364641+0000","last_change":"2024-05-27T08:35:04.015581+0000","last_active":"2024-05-27T08:35:58.364641+0000","last_peered":"2024-05-27T08:35:58.364641+0000","last_clean":"2024-05-27T08:35:58.364641+0000","last_became_active":"2024-05-27T08:35:04.015215+0000","last_became_peered":"2024-05-27T08:35:04.015215+0000","last_unstale":"2024-05-27T08:35:58.364641+0000","last_undegraded":"2024-05-27T08:35:58.364641+0000","last_fullsized":"2024-05-27T08:35:58.364641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:21:51.942008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367568+0000","last_change":"2024-05-27T08:35:07.049798+0000","last_active":"2024-05-27T08:35:58.367568+0000","last_peered":"2024-05-27T08:35:58.367568+0000","last_clean":"2024-05-27T08:35:58.367568+0000","last_became_active":"2024-05-27T08:35:07.048686+0000","last_became_peered":"2024-05-27T08:35:07.048686+0000","last_unstale":"2024-05-27T08:35:58.367568+0000","last_undegraded":"2024-05-27T08:35:58.367568+0000","last_fullsized":"2024-05-27T08:35:58.367568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:39:23.817581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374818+0000","last_change":"2024-05-27T08:35:04.023826+0000","last_active":"2024-05-27T08:35:58.374818+0000","last_peered":"2024-05-27T08:35:58.374818+0000","last_clean":"2024-05-27T08:35:58.374818+0000","last_became_active":"2024-05-27T08:35:04.023410+0000","last_became_peered":"2024-05-27T08:35:04.023410+0000","last_unstale":"2024-05-27T08:35:58.374818+0000","last_undegraded":"2024-05-27T08:35:58.374818+0000","last_fullsized":"2024-05-27T08:35:58.374818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:38:47.855186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368132+0000","last_change":"2024-05-27T08:35:07.038810+0000","last_active":"2024-05-27T08:35:58.368132+0000","last_peered":"2024-05-27T08:35:58.368132+0000","last_clean":"2024-05-27T08:35:58.368132+0000","last_became_active":"2024-05-27T08:35:07.038480+0000","last_became_peered":"2024-05-27T08:35:07.038480+0000","last_unstale":"2024-05-27T08:35:58.368132+0000","last_undegraded":"2024-05-27T08:35:58.368132+0000","last_fullsized":"2024-05-27T08:35:58.368132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:14.429081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":212,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:54.405504+0000","last_change":"2024-05-27T08:35:04.026934+0000","last_active":"2024-05-27T08:36:54.405504+0000","last_peered":"2024-05-27T08:36:54.405504+0000","last_clean":"2024-05-27T08:36:54.405504+0000","last_became_active":"2024-05-27T08:35:04.026577+0000","last_became_peered":"2024-05-27T08:35:04.026577+0000","last_unstale":"2024-05-27T08:36:54.405504+0000","last_undegraded":"2024-05-27T08:36:54.405504+0000","last_fullsized":"2024-05-27T08:36:54.405504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:59:56.067856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375300+0000","last_change":"2024-05-27T08:35:07.043840+0000","last_active":"2024-05-27T08:35:58.375300+0000","last_peered":"2024-05-27T08:35:58.375300+0000","last_clean":"2024-05-27T08:35:58.375300+0000","last_became_active":"2024-05-27T08:35:07.043224+0000","last_became_peered":"2024-05-27T08:35:07.043224+0000","last_unstale":"2024-05-27T08:35:58.375300+0000","last_undegraded":"2024-05-27T08:35:58.375300+0000","last_fullsized":"2024-05-27T08:35:58.375300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:20.212038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371395+0000","last_change":"2024-05-27T08:35:04.022602+0000","last_active":"2024-05-27T08:35:58.371395+0000","last_peered":"2024-05-27T08:35:58.371395+0000","last_clean":"2024-05-27T08:35:58.371395+0000","last_became_active":"2024-05-27T08:35:04.022203+0000","last_became_peered":"2024-05-27T08:35:04.022203+0000","last_unstale":"2024-05-27T08:35:58.371395+0000","last_undegraded":"2024-05-27T08:35:58.371395+0000","last_fullsized":"2024-05-27T08:35:58.371395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:51.490696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370927+0000","last_change":"2024-05-27T08:35:07.047257+0000","last_active":"2024-05-27T08:35:58.370927+0000","last_peered":"2024-05-27T08:35:58.370927+0000","last_clean":"2024-05-27T08:35:58.370927+0000","last_became_active":"2024-05-27T08:35:07.046639+0000","last_became_peered":"2024-05-27T08:35:07.046639+0000","last_unstale":"2024-05-27T08:35:58.370927+0000","last_undegraded":"2024-05-27T08:35:58.370927+0000","last_fullsized":"2024-05-27T08:35:58.370927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:04.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.376394+0000","last_change":"2024-05-27T08:35:04.245295+0000","last_active":"2024-05-27T08:35:58.376394+0000","last_peered":"2024-05-27T08:35:58.376394+0000","last_clean":"2024-05-27T08:35:58.376394+0000","last_became_active":"2024-05-27T08:35:04.244927+0000","last_became_peered":"2024-05-27T08:35:04.244927+0000","last_unstale":"2024-05-27T08:35:58.376394+0000","last_undegraded":"2024-05-27T08:35:58.376394+0000","last_fullsized":"2024-05-27T08:35:58.376394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:58.893480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374886+0000","last_change":"2024-05-27T08:35:07.048872+0000","last_active":"2024-05-27T08:35:58.374886+0000","last_peered":"2024-05-27T08:35:58.374886+0000","last_clean":"2024-05-27T08:35:58.374886+0000","last_became_active":"2024-05-27T08:35:07.048488+0000","last_became_peered":"2024-05-27T08:35:07.048488+0000","last_unstale":"2024-05-27T08:35:58.374886+0000","last_undegraded":"2024-05-27T08:35:58.374886+0000","last_fullsized":"2024-05-27T08:35:58.374886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:31.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":248,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:54.405361+0000","last_change":"2024-05-27T08:35:04.024718+0000","last_active":"2024-05-27T08:36:54.405361+0000","last_peered":"2024-05-27T08:36:54.405361+0000","last_clean":"2024-05-27T08:36:54.405361+0000","last_became_active":"2024-05-27T08:35:04.024155+0000","last_became_peered":"2024-05-27T08:35:04.024155+0000","last_unstale":"2024-05-27T08:36:54.405361+0000","last_undegraded":"2024-05-27T08:36:54.405361+0000","last_fullsized":"2024-05-27T08:36:54.405361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:22.573337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":153,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368315+0000","last_change":"2024-05-27T08:35:07.051342+0000","last_active":"2024-05-27T08:35:58.368315+0000","last_peered":"2024-05-27T08:35:58.368315+0000","last_clean":"2024-05-27T08:35:58.368315+0000","last_became_active":"2024-05-27T08:35:07.051030+0000","last_became_peered":"2024-05-27T08:35:07.051030+0000","last_unstale":"2024-05-27T08:35:58.368315+0000","last_undegraded":"2024-05-27T08:35:58.368315+0000","last_fullsized":"2024-05-27T08:35:58.368315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:48.245036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368633+0000","last_change":"2024-05-27T08:35:04.023717+0000","last_active":"2024-05-27T08:35:58.368633+0000","last_peered":"2024-05-27T08:35:58.368633+0000","last_clean":"2024-05-27T08:35:58.368633+0000","last_became_active":"2024-05-27T08:35:04.022017+0000","last_became_peered":"2024-05-27T08:35:04.022017+0000","last_unstale":"2024-05-27T08:35:58.368633+0000","last_undegraded":"2024-05-27T08:35:58.368633+0000","last_fullsized":"2024-05-27T08:35:58.368633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:55.834307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364601+0000","last_change":"2024-05-27T08:35:07.043980+0000","last_active":"2024-05-27T08:35:58.364601+0000","last_peered":"2024-05-27T08:35:58.364601+0000","last_clean":"2024-05-27T08:35:58.364601+0000","last_became_active":"2024-05-27T08:35:07.043672+0000","last_became_peered":"2024-05-27T08:35:07.043672+0000","last_unstale":"2024-05-27T08:35:58.364601+0000","last_undegraded":"2024-05-27T08:35:58.364601+0000","last_fullsized":"2024-05-27T08:35:58.364601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:03:30.243966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375453+0000","last_change":"2024-05-27T08:35:04.022731+0000","last_active":"2024-05-27T08:35:58.375453+0000","last_peered":"2024-05-27T08:35:58.375453+0000","last_clean":"2024-05-27T08:35:58.375453+0000","last_became_active":"2024-05-27T08:35:04.022349+0000","last_became_peered":"2024-05-27T08:35:04.022349+0000","last_unstale":"2024-05-27T08:35:58.375453+0000","last_undegraded":"2024-05-27T08:35:58.375453+0000","last_fullsized":"2024-05-27T08:35:58.375453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:38.651520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371218+0000","last_change":"2024-05-27T08:35:07.052623+0000","last_active":"2024-05-27T08:35:58.371218+0000","last_peered":"2024-05-27T08:35:58.371218+0000","last_clean":"2024-05-27T08:35:58.371218+0000","last_became_active":"2024-05-27T08:35:07.052313+0000","last_became_peered":"2024-05-27T08:35:07.052313+0000","last_unstale":"2024-05-27T08:35:58.371218+0000","last_undegraded":"2024-05-27T08:35:58.371218+0000","last_fullsized":"2024-05-27T08:35:58.371218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:24.612523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'6","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:52.403086+0000","last_change":"2024-05-27T08:35:04.023756+0000","last_active":"2024-05-27T08:36:52.403086+0000","last_peered":"2024-05-27T08:36:52.403086+0000","last_clean":"2024-05-27T08:36:52.403086+0000","last_became_active":"2024-05-27T08:35:04.023401+0000","last_became_peered":"2024-05-27T08:35:04.023401+0000","last_unstale":"2024-05-27T08:36:52.403086+0000","last_undegraded":"2024-05-27T08:36:52.403086+0000","last_fullsized":"2024-05-27T08:36:52.403086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:34.887246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":94,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370820+0000","last_change":"2024-05-27T08:35:07.050720+0000","last_active":"2024-05-27T08:35:58.370820+0000","last_peered":"2024-05-27T08:35:58.370820+0000","last_clean":"2024-05-27T08:35:58.370820+0000","last_became_active":"2024-05-27T08:35:07.050135+0000","last_became_peered":"2024-05-27T08:35:07.050135+0000","last_unstale":"2024-05-27T08:35:58.370820+0000","last_undegraded":"2024-05-27T08:35:58.370820+0000","last_fullsized":"2024-05-27T08:35:58.370820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:40.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.421002+0000","last_change":"2024-05-27T08:35:58.421002+0000","last_active":"2024-05-27T08:35:58.421002+0000","last_peered":"2024-05-27T08:35:58.421002+0000","last_clean":"2024-05-27T08:35:58.421002+0000","last_became_active":"2024-05-27T08:35:58.408889+0000","last_became_peered":"2024-05-27T08:35:58.408889+0000","last_unstale":"2024-05-27T08:35:58.421002+0000","last_undegraded":"2024-05-27T08:35:58.421002+0000","last_fullsized":"2024-05-27T08:35:58.421002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:29.705436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375224+0000","last_change":"2024-05-27T08:35:07.050504+0000","last_active":"2024-05-27T08:35:58.375224+0000","last_peered":"2024-05-27T08:35:58.375224+0000","last_clean":"2024-05-27T08:35:58.375224+0000","last_became_active":"2024-05-27T08:35:07.049368+0000","last_became_peered":"2024-05-27T08:35:07.049368+0000","last_unstale":"2024-05-27T08:35:58.375224+0000","last_undegraded":"2024-05-27T08:35:58.375224+0000","last_fullsized":"2024-05-27T08:35:58.375224+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:52.996313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368599+0000","last_change":"2024-05-27T08:35:04.023924+0000","last_active":"2024-05-27T08:35:58.368599+0000","last_peered":"2024-05-27T08:35:58.368599+0000","last_clean":"2024-05-27T08:35:58.368599+0000","last_became_active":"2024-05-27T08:35:04.023548+0000","last_became_peered":"2024-05-27T08:35:04.023548+0000","last_unstale":"2024-05-27T08:35:58.368599+0000","last_undegraded":"2024-05-27T08:35:58.368599+0000","last_fullsized":"2024-05-27T08:35:58.368599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:50.441490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.376335+0000","last_change":"2024-05-27T08:35:58.376335+0000","last_active":"2024-05-27T08:35:58.376335+0000","last_peered":"2024-05-27T08:35:58.376335+0000","last_clean":"2024-05-27T08:35:58.376335+0000","last_became_active":"2024-05-27T08:35:58.375926+0000","last_became_peered":"2024-05-27T08:35:58.375926+0000","last_unstale":"2024-05-27T08:35:58.376335+0000","last_undegraded":"2024-05-27T08:35:58.376335+0000","last_fullsized":"2024-05-27T08:35:58.376335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:33.892172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374310+0000","last_change":"2024-05-27T08:35:58.374310+0000","last_active":"2024-05-27T08:35:58.374310+0000","last_peered":"2024-05-27T08:35:58.374310+0000","last_clean":"2024-05-27T08:35:58.374310+0000","last_became_active":"2024-05-27T08:35:58.373929+0000","last_became_peered":"2024-05-27T08:35:58.373929+0000","last_unstale":"2024-05-27T08:35:58.374310+0000","last_undegraded":"2024-05-27T08:35:58.374310+0000","last_fullsized":"2024-05-27T08:35:58.374310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:20.901125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367829+0000","last_change":"2024-05-27T08:35:07.039182+0000","last_active":"2024-05-27T08:35:58.367829+0000","last_peered":"2024-05-27T08:35:58.367829+0000","last_clean":"2024-05-27T08:35:58.367829+0000","last_became_active":"2024-05-27T08:35:07.038793+0000","last_became_peered":"2024-05-27T08:35:07.038793+0000","last_unstale":"2024-05-27T08:35:58.367829+0000","last_undegraded":"2024-05-27T08:35:58.367829+0000","last_fullsized":"2024-05-27T08:35:58.367829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:03.839681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":119,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:50.443425+0000","last_change":"2024-05-27T08:35:04.023194+0000","last_active":"2024-05-27T08:36:50.443425+0000","last_peered":"2024-05-27T08:36:50.443425+0000","last_clean":"2024-05-27T08:36:50.443425+0000","last_became_active":"2024-05-27T08:35:04.022823+0000","last_became_peered":"2024-05-27T08:35:04.022823+0000","last_unstale":"2024-05-27T08:36:50.443425+0000","last_undegraded":"2024-05-27T08:36:50.443425+0000","last_fullsized":"2024-05-27T08:36:50.443425+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:44:28.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":89,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367676+0000","last_change":"2024-05-27T08:35:07.050717+0000","last_active":"2024-05-27T08:35:58.367676+0000","last_peered":"2024-05-27T08:35:58.367676+0000","last_clean":"2024-05-27T08:35:58.367676+0000","last_became_active":"2024-05-27T08:35:07.050413+0000","last_became_peered":"2024-05-27T08:35:07.050413+0000","last_unstale":"2024-05-27T08:35:58.367676+0000","last_undegraded":"2024-05-27T08:35:58.367676+0000","last_fullsized":"2024-05-27T08:35:58.367676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:00:27.693141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375745+0000","last_change":"2024-05-27T08:35:07.053614+0000","last_active":"2024-05-27T08:35:58.375745+0000","last_peered":"2024-05-27T08:35:58.375745+0000","last_clean":"2024-05-27T08:35:58.375745+0000","last_became_active":"2024-05-27T08:35:07.053253+0000","last_became_peered":"2024-05-27T08:35:07.053253+0000","last_unstale":"2024-05-27T08:35:58.375745+0000","last_undegraded":"2024-05-27T08:35:58.375745+0000","last_fullsized":"2024-05-27T08:35:58.375745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:15:08.985984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"83'5","reported_seq":121,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:51.400942+0000","last_change":"2024-05-27T08:35:04.246500+0000","last_active":"2024-05-27T08:36:51.400942+0000","last_peered":"2024-05-27T08:36:51.400942+0000","last_clean":"2024-05-27T08:36:51.400942+0000","last_became_active":"2024-05-27T08:35:04.244575+0000","last_became_peered":"2024-05-27T08:35:04.244575+0000","last_unstale":"2024-05-27T08:36:51.400942+0000","last_undegraded":"2024-05-27T08:36:51.400942+0000","last_fullsized":"2024-05-27T08:36:51.400942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:24:10.818057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368091+0000","last_change":"2024-05-27T08:35:07.048488+0000","last_active":"2024-05-27T08:35:58.368091+0000","last_peered":"2024-05-27T08:35:58.368091+0000","last_clean":"2024-05-27T08:35:58.368091+0000","last_became_active":"2024-05-27T08:35:07.048172+0000","last_became_peered":"2024-05-27T08:35:07.048172+0000","last_unstale":"2024-05-27T08:35:58.368091+0000","last_undegraded":"2024-05-27T08:35:58.368091+0000","last_fullsized":"2024-05-27T08:35:58.368091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:01.632944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.407485+0000","last_change":"2024-05-27T08:35:04.022807+0000","last_active":"2024-05-27T08:35:58.407485+0000","last_peered":"2024-05-27T08:35:58.407485+0000","last_clean":"2024-05-27T08:35:58.407485+0000","last_became_active":"2024-05-27T08:35:04.021198+0000","last_became_peered":"2024-05-27T08:35:04.021198+0000","last_unstale":"2024-05-27T08:35:58.407485+0000","last_undegraded":"2024-05-27T08:35:58.407485+0000","last_fullsized":"2024-05-27T08:35:58.407485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:42:10.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":101,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370999+0000","last_change":"2024-05-27T08:33:45.683600+0000","last_active":"2024-05-27T08:35:58.370999+0000","last_peered":"2024-05-27T08:35:58.370999+0000","last_clean":"2024-05-27T08:35:58.370999+0000","last_became_active":"2024-05-27T08:33:45.677989+0000","last_became_peered":"2024-05-27T08:33:45.677989+0000","last_unstale":"2024-05-27T08:35:58.370999+0000","last_undegraded":"2024-05-27T08:35:58.370999+0000","last_fullsized":"2024-05-27T08:35:58.370999+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_clean_scrub_stamp":"2024-05-27T08:30:25.507727+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:48:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364496+0000","last_change":"2024-05-27T08:35:07.045879+0000","last_active":"2024-05-27T08:35:58.364496+0000","last_peered":"2024-05-27T08:35:58.364496+0000","last_clean":"2024-05-27T08:35:58.364496+0000","last_became_active":"2024-05-27T08:35:07.045522+0000","last_became_peered":"2024-05-27T08:35:07.045522+0000","last_unstale":"2024-05-27T08:35:58.364496+0000","last_undegraded":"2024-05-27T08:35:58.364496+0000","last_fullsized":"2024-05-27T08:35:58.364496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:30.393769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371043+0000","last_change":"2024-05-27T08:35:04.022419+0000","last_active":"2024-05-27T08:35:58.371043+0000","last_peered":"2024-05-27T08:35:58.371043+0000","last_clean":"2024-05-27T08:35:58.371043+0000","last_became_active":"2024-05-27T08:35:04.022039+0000","last_became_peered":"2024-05-27T08:35:04.022039+0000","last_unstale":"2024-05-27T08:35:58.371043+0000","last_undegraded":"2024-05-27T08:35:58.371043+0000","last_fullsized":"2024-05-27T08:35:58.371043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:31.260596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368426+0000","last_change":"2024-05-27T08:35:04.245214+0000","last_active":"2024-05-27T08:35:58.368426+0000","last_peered":"2024-05-27T08:35:58.368426+0000","last_clean":"2024-05-27T08:35:58.368426+0000","last_became_active":"2024-05-27T08:35:04.244768+0000","last_became_peered":"2024-05-27T08:35:04.244768+0000","last_unstale":"2024-05-27T08:35:58.368426+0000","last_undegraded":"2024-05-27T08:35:58.368426+0000","last_fullsized":"2024-05-27T08:35:58.368426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:34.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364778+0000","last_change":"2024-05-27T08:35:07.042721+0000","last_active":"2024-05-27T08:35:58.364778+0000","last_peered":"2024-05-27T08:35:58.364778+0000","last_clean":"2024-05-27T08:35:58.364778+0000","last_became_active":"2024-05-27T08:35:07.042435+0000","last_became_peered":"2024-05-27T08:35:07.042435+0000","last_unstale":"2024-05-27T08:35:58.364778+0000","last_undegraded":"2024-05-27T08:35:58.364778+0000","last_fullsized":"2024-05-27T08:35:58.364778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:49.186753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368668+0000","last_change":"2024-05-27T08:35:04.017995+0000","last_active":"2024-05-27T08:35:58.368668+0000","last_peered":"2024-05-27T08:35:58.368668+0000","last_clean":"2024-05-27T08:35:58.368668+0000","last_became_active":"2024-05-27T08:35:04.017675+0000","last_became_peered":"2024-05-27T08:35:04.017675+0000","last_unstale":"2024-05-27T08:35:58.368668+0000","last_undegraded":"2024-05-27T08:35:58.368668+0000","last_fullsized":"2024-05-27T08:35:58.368668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:23:15.452624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368276+0000","last_change":"2024-05-27T08:35:07.046553+0000","last_active":"2024-05-27T08:35:58.368276+0000","last_peered":"2024-05-27T08:35:58.368276+0000","last_clean":"2024-05-27T08:35:58.368276+0000","last_became_active":"2024-05-27T08:35:07.046229+0000","last_became_peered":"2024-05-27T08:35:07.046229+0000","last_unstale":"2024-05-27T08:35:58.368276+0000","last_undegraded":"2024-05-27T08:35:58.368276+0000","last_fullsized":"2024-05-27T08:35:58.368276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:21.024307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.407482+0000","last_change":"2024-05-27T08:35:04.262439+0000","last_active":"2024-05-27T08:35:58.407482+0000","last_peered":"2024-05-27T08:35:58.407482+0000","last_clean":"2024-05-27T08:35:58.407482+0000","last_became_active":"2024-05-27T08:35:04.261958+0000","last_became_peered":"2024-05-27T08:35:04.261958+0000","last_unstale":"2024-05-27T08:35:58.407482+0000","last_undegraded":"2024-05-27T08:35:58.407482+0000","last_fullsized":"2024-05-27T08:35:58.407482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:11:05.763068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370773+0000","last_change":"2024-05-27T08:35:07.042307+0000","last_active":"2024-05-27T08:35:58.370773+0000","last_peered":"2024-05-27T08:35:58.370773+0000","last_clean":"2024-05-27T08:35:58.370773+0000","last_became_active":"2024-05-27T08:35:07.042001+0000","last_became_peered":"2024-05-27T08:35:07.042001+0000","last_unstale":"2024-05-27T08:35:58.370773+0000","last_undegraded":"2024-05-27T08:35:58.370773+0000","last_fullsized":"2024-05-27T08:35:58.370773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:51.030157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374562+0000","last_change":"2024-05-27T08:35:04.245436+0000","last_active":"2024-05-27T08:35:58.374562+0000","last_peered":"2024-05-27T08:35:58.374562+0000","last_clean":"2024-05-27T08:35:58.374562+0000","last_became_active":"2024-05-27T08:35:04.245039+0000","last_became_peered":"2024-05-27T08:35:04.245039+0000","last_unstale":"2024-05-27T08:35:58.374562+0000","last_undegraded":"2024-05-27T08:35:58.374562+0000","last_fullsized":"2024-05-27T08:35:58.374562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:10:11.756864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367831+0000","last_change":"2024-05-27T08:35:07.051524+0000","last_active":"2024-05-27T08:35:58.367831+0000","last_peered":"2024-05-27T08:35:58.367831+0000","last_clean":"2024-05-27T08:35:58.367831+0000","last_became_active":"2024-05-27T08:35:07.050676+0000","last_became_peered":"2024-05-27T08:35:07.050676+0000","last_unstale":"2024-05-27T08:35:58.367831+0000","last_undegraded":"2024-05-27T08:35:58.367831+0000","last_fullsized":"2024-05-27T08:35:58.367831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:06.818977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367648+0000","last_change":"2024-05-27T08:35:04.022064+0000","last_active":"2024-05-27T08:35:58.367648+0000","last_peered":"2024-05-27T08:35:58.367648+0000","last_clean":"2024-05-27T08:35:58.367648+0000","last_became_active":"2024-05-27T08:35:04.021691+0000","last_became_peered":"2024-05-27T08:35:04.021691+0000","last_unstale":"2024-05-27T08:35:58.367648+0000","last_undegraded":"2024-05-27T08:35:58.367648+0000","last_fullsized":"2024-05-27T08:35:58.367648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:54.557756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371173+0000","last_change":"2024-05-27T08:35:07.049666+0000","last_active":"2024-05-27T08:35:58.371173+0000","last_peered":"2024-05-27T08:35:58.371173+0000","last_clean":"2024-05-27T08:35:58.371173+0000","last_became_active":"2024-05-27T08:35:07.049312+0000","last_became_peered":"2024-05-27T08:35:07.049312+0000","last_unstale":"2024-05-27T08:35:58.371173+0000","last_undegraded":"2024-05-27T08:35:58.371173+0000","last_fullsized":"2024-05-27T08:35:58.371173+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:37:53.704582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368318+0000","last_change":"2024-05-27T08:35:04.018457+0000","last_active":"2024-05-27T08:35:58.368318+0000","last_peered":"2024-05-27T08:35:58.368318+0000","last_clean":"2024-05-27T08:35:58.368318+0000","last_became_active":"2024-05-27T08:35:04.018008+0000","last_became_peered":"2024-05-27T08:35:04.018008+0000","last_unstale":"2024-05-27T08:35:58.368318+0000","last_undegraded":"2024-05-27T08:35:58.368318+0000","last_fullsized":"2024-05-27T08:35:58.368318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:08:13.228725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367962+0000","last_change":"2024-05-27T08:35:07.047040+0000","last_active":"2024-05-27T08:35:58.367962+0000","last_peered":"2024-05-27T08:35:58.367962+0000","last_clean":"2024-05-27T08:35:58.367962+0000","last_became_active":"2024-05-27T08:35:07.046731+0000","last_became_peered":"2024-05-27T08:35:07.046731+0000","last_unstale":"2024-05-27T08:35:58.367962+0000","last_undegraded":"2024-05-27T08:35:58.367962+0000","last_fullsized":"2024-05-27T08:35:58.367962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:25.368349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368147+0000","last_change":"2024-05-27T08:35:04.022804+0000","last_active":"2024-05-27T08:35:58.368147+0000","last_peered":"2024-05-27T08:35:58.368147+0000","last_clean":"2024-05-27T08:35:58.368147+0000","last_became_active":"2024-05-27T08:35:04.022458+0000","last_became_peered":"2024-05-27T08:35:04.022458+0000","last_unstale":"2024-05-27T08:35:58.368147+0000","last_undegraded":"2024-05-27T08:35:58.368147+0000","last_fullsized":"2024-05-27T08:35:58.368147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:01.029226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371185+0000","last_change":"2024-05-27T08:35:07.049136+0000","last_active":"2024-05-27T08:35:58.371185+0000","last_peered":"2024-05-27T08:35:58.371185+0000","last_clean":"2024-05-27T08:35:58.371185+0000","last_became_active":"2024-05-27T08:35:07.048793+0000","last_became_peered":"2024-05-27T08:35:07.048793+0000","last_unstale":"2024-05-27T08:35:58.371185+0000","last_undegraded":"2024-05-27T08:35:58.371185+0000","last_fullsized":"2024-05-27T08:35:58.371185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:10:21.191743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":122,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:52.403534+0000","last_change":"2024-05-27T08:35:04.262102+0000","last_active":"2024-05-27T08:36:52.403534+0000","last_peered":"2024-05-27T08:36:52.403534+0000","last_clean":"2024-05-27T08:36:52.403534+0000","last_became_active":"2024-05-27T08:35:04.261665+0000","last_became_peered":"2024-05-27T08:35:04.261665+0000","last_unstale":"2024-05-27T08:36:52.403534+0000","last_undegraded":"2024-05-27T08:36:52.403534+0000","last_fullsized":"2024-05-27T08:36:52.403534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:57:57.712197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371431+0000","last_change":"2024-05-27T08:35:07.051557+0000","last_active":"2024-05-27T08:35:58.371431+0000","last_peered":"2024-05-27T08:35:58.371431+0000","last_clean":"2024-05-27T08:35:58.371431+0000","last_became_active":"2024-05-27T08:35:07.051246+0000","last_became_peered":"2024-05-27T08:35:07.051246+0000","last_unstale":"2024-05-27T08:35:58.371431+0000","last_undegraded":"2024-05-27T08:35:58.371431+0000","last_fullsized":"2024-05-27T08:35:58.371431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:08:13.248301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.376106+0000","last_change":"2024-05-27T08:35:04.022328+0000","last_active":"2024-05-27T08:35:58.376106+0000","last_peered":"2024-05-27T08:35:58.376106+0000","last_clean":"2024-05-27T08:35:58.376106+0000","last_became_active":"2024-05-27T08:35:04.021711+0000","last_became_peered":"2024-05-27T08:35:04.021711+0000","last_unstale":"2024-05-27T08:35:58.376106+0000","last_undegraded":"2024-05-27T08:35:58.376106+0000","last_fullsized":"2024-05-27T08:35:58.376106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:32.841474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371211+0000","last_change":"2024-05-27T08:35:07.056768+0000","last_active":"2024-05-27T08:35:58.371211+0000","last_peered":"2024-05-27T08:35:58.371211+0000","last_clean":"2024-05-27T08:35:58.371211+0000","last_became_active":"2024-05-27T08:35:07.056482+0000","last_became_peered":"2024-05-27T08:35:07.056482+0000","last_unstale":"2024-05-27T08:35:58.371211+0000","last_undegraded":"2024-05-27T08:35:58.371211+0000","last_fullsized":"2024-05-27T08:35:58.371211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:23.515625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364744+0000","last_change":"2024-05-27T08:35:04.016613+0000","last_active":"2024-05-27T08:35:58.364744+0000","last_peered":"2024-05-27T08:35:58.364744+0000","last_clean":"2024-05-27T08:35:58.364744+0000","last_became_active":"2024-05-27T08:35:04.016268+0000","last_became_peered":"2024-05-27T08:35:04.016268+0000","last_unstale":"2024-05-27T08:35:58.364744+0000","last_undegraded":"2024-05-27T08:35:58.364744+0000","last_fullsized":"2024-05-27T08:35:58.364744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:11.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371530+0000","last_change":"2024-05-27T08:35:07.051515+0000","last_active":"2024-05-27T08:35:58.371530+0000","last_peered":"2024-05-27T08:35:58.371530+0000","last_clean":"2024-05-27T08:35:58.371530+0000","last_became_active":"2024-05-27T08:35:07.051197+0000","last_became_peered":"2024-05-27T08:35:07.051197+0000","last_unstale":"2024-05-27T08:35:58.371530+0000","last_undegraded":"2024-05-27T08:35:58.371530+0000","last_fullsized":"2024-05-27T08:35:58.371530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:49.233474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371298+0000","last_change":"2024-05-27T08:35:07.051960+0000","last_active":"2024-05-27T08:35:58.371298+0000","last_peered":"2024-05-27T08:35:58.371298+0000","last_clean":"2024-05-27T08:35:58.371298+0000","last_became_active":"2024-05-27T08:35:07.051395+0000","last_became_peered":"2024-05-27T08:35:07.051395+0000","last_unstale":"2024-05-27T08:35:58.371298+0000","last_undegraded":"2024-05-27T08:35:58.371298+0000","last_fullsized":"2024-05-27T08:35:58.371298+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:57.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368861+0000","last_change":"2024-05-27T08:35:07.050096+0000","last_active":"2024-05-27T08:35:58.368861+0000","last_peered":"2024-05-27T08:35:58.368861+0000","last_clean":"2024-05-27T08:35:58.368861+0000","last_became_active":"2024-05-27T08:35:07.049755+0000","last_became_peered":"2024-05-27T08:35:07.049755+0000","last_unstale":"2024-05-27T08:35:58.368861+0000","last_undegraded":"2024-05-27T08:35:58.368861+0000","last_fullsized":"2024-05-27T08:35:58.368861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:39.768128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368191+0000","last_change":"2024-05-27T08:35:07.048281+0000","last_active":"2024-05-27T08:35:58.368191+0000","last_peered":"2024-05-27T08:35:58.368191+0000","last_clean":"2024-05-27T08:35:58.368191+0000","last_became_active":"2024-05-27T08:35:07.047879+0000","last_became_peered":"2024-05-27T08:35:07.047879+0000","last_unstale":"2024-05-27T08:35:58.368191+0000","last_undegraded":"2024-05-27T08:35:58.368191+0000","last_fullsized":"2024-05-27T08:35:58.368191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:43.631671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370546+0000","last_change":"2024-05-27T08:35:07.050501+0000","last_active":"2024-05-27T08:35:58.370546+0000","last_peered":"2024-05-27T08:35:58.370546+0000","last_clean":"2024-05-27T08:35:58.370546+0000","last_became_active":"2024-05-27T08:35:07.050159+0000","last_became_peered":"2024-05-27T08:35:07.050159+0000","last_unstale":"2024-05-27T08:35:58.370546+0000","last_undegraded":"2024-05-27T08:35:58.370546+0000","last_fullsized":"2024-05-27T08:35:58.370546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:17:23.293019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.372446+0000","last_change":"2024-05-27T08:35:58.372446+0000","last_active":"2024-05-27T08:35:58.372446+0000","last_peered":"2024-05-27T08:35:58.372446+0000","last_clean":"2024-05-27T08:35:58.372446+0000","last_became_active":"2024-05-27T08:35:58.372080+0000","last_became_peered":"2024-05-27T08:35:58.372080+0000","last_unstale":"2024-05-27T08:35:58.372446+0000","last_undegraded":"2024-05-27T08:35:58.372446+0000","last_fullsized":"2024-05-27T08:35:58.372446+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:57.828720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371674+0000","last_change":"2024-05-27T08:35:07.052382+0000","last_active":"2024-05-27T08:35:58.371674+0000","last_peered":"2024-05-27T08:35:58.371674+0000","last_clean":"2024-05-27T08:35:58.371674+0000","last_became_active":"2024-05-27T08:35:07.052049+0000","last_became_peered":"2024-05-27T08:35:07.052049+0000","last_unstale":"2024-05-27T08:35:58.371674+0000","last_undegraded":"2024-05-27T08:35:58.371674+0000","last_fullsized":"2024-05-27T08:35:58.371674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:08.912872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.369555+0000","last_change":"2024-05-27T08:35:07.043810+0000","last_active":"2024-05-27T08:35:58.369555+0000","last_peered":"2024-05-27T08:35:58.369555+0000","last_clean":"2024-05-27T08:35:58.369555+0000","last_became_active":"2024-05-27T08:35:07.043449+0000","last_became_peered":"2024-05-27T08:35:07.043449+0000","last_unstale":"2024-05-27T08:35:58.369555+0000","last_undegraded":"2024-05-27T08:35:58.369555+0000","last_fullsized":"2024-05-27T08:35:58.369555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:18.089405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371407+0000","last_change":"2024-05-27T08:35:58.371407+0000","last_active":"2024-05-27T08:35:58.371407+0000","last_peered":"2024-05-27T08:35:58.371407+0000","last_clean":"2024-05-27T08:35:58.371407+0000","last_became_active":"2024-05-27T08:35:58.370963+0000","last_became_peered":"2024-05-27T08:35:58.370963+0000","last_unstale":"2024-05-27T08:35:58.371407+0000","last_undegraded":"2024-05-27T08:35:58.371407+0000","last_fullsized":"2024-05-27T08:35:58.371407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:23.458573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375512+0000","last_change":"2024-05-27T08:35:07.049960+0000","last_active":"2024-05-27T08:35:58.375512+0000","last_peered":"2024-05-27T08:35:58.375512+0000","last_clean":"2024-05-27T08:35:58.375512+0000","last_became_active":"2024-05-27T08:35:07.049595+0000","last_became_peered":"2024-05-27T08:35:07.049595+0000","last_unstale":"2024-05-27T08:35:58.375512+0000","last_undegraded":"2024-05-27T08:35:58.375512+0000","last_fullsized":"2024-05-27T08:35:58.375512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:18.182658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367599+0000","last_change":"2024-05-27T08:35:07.043790+0000","last_active":"2024-05-27T08:35:58.367599+0000","last_peered":"2024-05-27T08:35:58.367599+0000","last_clean":"2024-05-27T08:35:58.367599+0000","last_became_active":"2024-05-27T08:35:07.043476+0000","last_became_peered":"2024-05-27T08:35:07.043476+0000","last_unstale":"2024-05-27T08:35:58.367599+0000","last_undegraded":"2024-05-27T08:35:58.367599+0000","last_fullsized":"2024-05-27T08:35:58.367599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:01.626979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374792+0000","last_change":"2024-05-27T08:35:07.044928+0000","last_active":"2024-05-27T08:35:58.374792+0000","last_peered":"2024-05-27T08:35:58.374792+0000","last_clean":"2024-05-27T08:35:58.374792+0000","last_became_active":"2024-05-27T08:35:07.044577+0000","last_became_peered":"2024-05-27T08:35:07.044577+0000","last_unstale":"2024-05-27T08:35:58.374792+0000","last_undegraded":"2024-05-27T08:35:58.374792+0000","last_fullsized":"2024-05-27T08:35:58.374792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:36.560525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364803+0000","last_change":"2024-05-27T08:35:07.045067+0000","last_active":"2024-05-27T08:35:58.364803+0000","last_peered":"2024-05-27T08:35:58.364803+0000","last_clean":"2024-05-27T08:35:58.364803+0000","last_became_active":"2024-05-27T08:35:07.044759+0000","last_became_peered":"2024-05-27T08:35:07.044759+0000","last_unstale":"2024-05-27T08:35:58.364803+0000","last_undegraded":"2024-05-27T08:35:58.364803+0000","last_fullsized":"2024-05-27T08:35:58.364803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:08.440135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.373032+0000","last_change":"2024-05-27T08:35:07.048186+0000","last_active":"2024-05-27T08:35:58.373032+0000","last_peered":"2024-05-27T08:35:58.373032+0000","last_clean":"2024-05-27T08:35:58.373032+0000","last_became_active":"2024-05-27T08:35:07.047828+0000","last_became_peered":"2024-05-27T08:35:07.047828+0000","last_unstale":"2024-05-27T08:35:58.373032+0000","last_undegraded":"2024-05-27T08:35:58.373032+0000","last_fullsized":"2024-05-27T08:35:58.373032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:09.928234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66966,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":934,"num_read_kb":538,"num_write":246,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":200898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":448717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1099317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":448065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135924,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":447189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":1041352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":447782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":509129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":449121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":508956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":447607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":448146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712444,"statfs":{"total":95995035648,"available":95961542656,"internally_reserved":0,"allocated":1564672,"data_stored":1040639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T08:36:57.044 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T08:36:57.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:56 smithi179 ceph-mon[36481]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:57.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:56 smithi092 ceph-mon[36049]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:36:57.309 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:36:58.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:57 smithi179 ceph-mon[36481]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:36:58.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:57 smithi077 ceph-mon[32007]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:36:58.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:57 smithi092 ceph-mon[36049]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3833925025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806bb70 msgr2=0x7f2d1810fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3833925025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806bb70 0x7f2d1810fab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009990 tx=0x7f2d0c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3833925025 shutdown_connections 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3833925025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d1810fff0 0x7f2d181123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3833925025 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1806b250 0x7f2d1806b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3833925025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806bb70 0x7f2d1810fab0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3833925025 >> 172.21.15.77:0/3833925025 conn(0x7f2d180793f0 msgr2=0x7f2d18079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3833925025 shutdown_connections 2024-05-27T08:36:59.054 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.052+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3833925025 wait complete. 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1cd68700 1 Processor -- start 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1cd68700 1 -- start start 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1cd68700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806b250 0x7f2d1819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1cd68700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1806bb70 0x7f2d1819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1cd68700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d1810fff0 0x7f2d181a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1cd68700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18115900 con 0x7f2d1806b250 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1cd68700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18115780 con 0x7f2d1806bb70 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1cd68700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18115a80 con 0x7f2d1810fff0 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806b250 0x7f2d1819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806b250 0x7f2d1819be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:56874/0 (socket says 172.21.15.77:56874) 2024-05-27T08:36:59.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d1659c700 1 -- 172.21.15.77:0/3263981559 learned_addr learned my addr 172.21.15.77:0/3263981559 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d15d9b700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1806bb70 0x7f2d1819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.053+0000 7f2d16d9d700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d1810fff0 0x7f2d181a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d1659c700 1 -- 172.21.15.77:0/3263981559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d1810fff0 msgr2=0x7f2d181a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d1659c700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d1810fff0 0x7f2d181a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d1659c700 1 -- 172.21.15.77:0/3263981559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1806bb70 msgr2=0x7f2d1819c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d1659c700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1806bb70 0x7f2d1819c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d1659c700 1 -- 172.21.15.77:0/3263981559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d181a0df0 con 0x7f2d1806b250 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d1659c700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806b250 0x7f2d1819be30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800d980 tx=0x7f2d0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d077fe700 1 -- 172.21.15.77:0/3263981559 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08016070 con 0x7f2d1806b250 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d077fe700 1 -- 172.21.15.77:0/3263981559 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d08004830 con 0x7f2d1806b250 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d181a10e0 con 0x7f2d1806b250 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d077fe700 1 -- 172.21.15.77:0/3263981559 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08005540 con 0x7f2d1806b250 2024-05-27T08:36:59.057 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.054+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181a8960 con 0x7f2d1806b250 2024-05-27T08:36:59.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.055+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1806a230 con 0x7f2d1806b250 2024-05-27T08:36:59.058 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.057+0000 7f2d077fe700 1 -- 172.21.15.77:0/3263981559 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d08004220 con 0x7f2d1806b250 2024-05-27T08:36:59.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.058+0000 7f2d077fe700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d000b2880 0x7f2d000b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:36:59.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.058+0000 7f2d15d9b700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d000b2880 0x7f2d000b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:36:59.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.058+0000 7f2d077fe700 1 -- 172.21.15.77:0/3263981559 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2d080e2530 con 0x7f2d1806b250 2024-05-27T08:36:59.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.058+0000 7f2d15d9b700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d000b2880 0x7f2d000b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00b5c0 tx=0x7f2d0c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:36:59.063 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.062+0000 7f2d077fe700 1 -- 172.21.15.77:0/3263981559 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d0801f020 con 0x7f2d1806b250 2024-05-27T08:36:59.064 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:58 smithi077 ceph-mon[32007]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:59.064 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:36:58 smithi077 ceph-mon[32007]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-05-27T08:36:59.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:58 smithi179 ceph-mon[36481]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:59.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:36:58 smithi179 ceph-mon[36481]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-05-27T08:36:59.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:58 smithi092 ceph-mon[36049]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:36:59.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:36:58 smithi092 ceph-mon[36049]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-05-27T08:36:59.264 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.263+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d18063980 con 0x7f2d000b2880 2024-05-27T08:36:59.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.270+0000 7f2d077fe700 1 -- 172.21.15.77:0/3263981559 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258920 (secure 0 0 0) 0x7f2d18063980 con 0x7f2d000b2880 2024-05-27T08:36:59.275 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:36:59.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.273+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d000b2880 msgr2=0x7f2d000b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:59.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.273+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d000b2880 0x7f2d000b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00b5c0 tx=0x7f2d0c005960 comp rx=0 tx=0).stop 2024-05-27T08:36:59.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.273+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806b250 msgr2=0x7f2d1819be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:36:59.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.273+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806b250 0x7f2d1819be30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800d980 tx=0x7f2d0800dc90 comp rx=0 tx=0).stop 2024-05-27T08:36:59.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.273+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 shutdown_connections 2024-05-27T08:36:59.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.274+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2d000b2880 0x7f2d000b4d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.274+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d1810fff0 0x7f2d181a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.274+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d1806b250 0x7f2d1819be30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.274+0000 7f2d1cd68700 1 --2- 172.21.15.77:0/3263981559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d1806bb70 0x7f2d1819c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:36:59.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.274+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 >> 172.21.15.77:0/3263981559 conn(0x7f2d180793f0 msgr2=0x7f2d181114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:36:59.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.274+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 shutdown_connections 2024-05-27T08:36:59.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:36:59.274+0000 7f2d1cd68700 1 -- 172.21.15.77:0/3263981559 wait complete. 2024-05-27T08:36:59.278 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-05-27T08:37:00.033 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-05-27T08:36:59.178821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657850,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1042,"num_read_kb":619,"num_write":374,"num_write_kb":2290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2953206,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":560,"ondisk_log_size":560,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366456,"kb_used_data":13116,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575368,"statfs":{"total":1151940427776,"available":1151565176832,"internally_reserved":0,"allocated":13430784,"data_stored":7336302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":282,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":62,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006043"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.188071+0000","last_change":"2024-05-27T08:35:07.046361+0000","last_active":"2024-05-27T08:36:57.188071+0000","last_peered":"2024-05-27T08:36:57.188071+0000","last_clean":"2024-05-27T08:36:57.188071+0000","last_became_active":"2024-05-27T08:35:07.045954+0000","last_became_peered":"2024-05-27T08:35:07.045954+0000","last_unstale":"2024-05-27T08:36:57.188071+0000","last_undegraded":"2024-05-27T08:36:57.188071+0000","last_fullsized":"2024-05-27T08:36:57.188071+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:03:47.458964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370683+0000","last_change":"2024-05-27T08:35:07.043725+0000","last_active":"2024-05-27T08:35:58.370683+0000","last_peered":"2024-05-27T08:35:58.370683+0000","last_clean":"2024-05-27T08:35:58.370683+0000","last_became_active":"2024-05-27T08:35:07.043447+0000","last_became_peered":"2024-05-27T08:35:07.043447+0000","last_unstale":"2024-05-27T08:35:58.370683+0000","last_undegraded":"2024-05-27T08:35:58.370683+0000","last_fullsized":"2024-05-27T08:35:58.370683+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:44.509202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370533+0000","last_change":"2024-05-27T08:35:07.048075+0000","last_active":"2024-05-27T08:35:58.370533+0000","last_peered":"2024-05-27T08:35:58.370533+0000","last_clean":"2024-05-27T08:35:58.370533+0000","last_became_active":"2024-05-27T08:35:07.047687+0000","last_became_peered":"2024-05-27T08:35:07.047687+0000","last_unstale":"2024-05-27T08:35:58.370533+0000","last_undegraded":"2024-05-27T08:35:58.370533+0000","last_fullsized":"2024-05-27T08:35:58.370533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:05:15.197231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.167191+0000","last_change":"2024-05-27T08:35:07.046809+0000","last_active":"2024-05-27T08:36:57.167191+0000","last_peered":"2024-05-27T08:36:57.167191+0000","last_clean":"2024-05-27T08:36:57.167191+0000","last_became_active":"2024-05-27T08:35:07.046453+0000","last_became_peered":"2024-05-27T08:35:07.046453+0000","last_unstale":"2024-05-27T08:36:57.167191+0000","last_undegraded":"2024-05-27T08:36:57.167191+0000","last_fullsized":"2024-05-27T08:36:57.167191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:43.209613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.169270+0000","last_change":"2024-05-27T08:35:07.040242+0000","last_active":"2024-05-27T08:36:57.169270+0000","last_peered":"2024-05-27T08:36:57.169270+0000","last_clean":"2024-05-27T08:36:57.169270+0000","last_became_active":"2024-05-27T08:35:07.039860+0000","last_became_peered":"2024-05-27T08:35:07.039860+0000","last_unstale":"2024-05-27T08:36:57.169270+0000","last_undegraded":"2024-05-27T08:36:57.169270+0000","last_fullsized":"2024-05-27T08:36:57.169270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:49.212799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367633+0000","last_change":"2024-05-27T08:35:07.048176+0000","last_active":"2024-05-27T08:35:58.367633+0000","last_peered":"2024-05-27T08:35:58.367633+0000","last_clean":"2024-05-27T08:35:58.367633+0000","last_became_active":"2024-05-27T08:35:07.047860+0000","last_became_peered":"2024-05-27T08:35:07.047860+0000","last_unstale":"2024-05-27T08:35:58.367633+0000","last_undegraded":"2024-05-27T08:35:58.367633+0000","last_fullsized":"2024-05-27T08:35:58.367633+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:19.459704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370920+0000","last_change":"2024-05-27T08:35:07.052285+0000","last_active":"2024-05-27T08:35:58.370920+0000","last_peered":"2024-05-27T08:35:58.370920+0000","last_clean":"2024-05-27T08:35:58.370920+0000","last_became_active":"2024-05-27T08:35:07.051970+0000","last_became_peered":"2024-05-27T08:35:07.051970+0000","last_unstale":"2024-05-27T08:35:58.370920+0000","last_undegraded":"2024-05-27T08:35:58.370920+0000","last_fullsized":"2024-05-27T08:35:58.370920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:56.543022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.176070+0000","last_change":"2024-05-27T08:35:07.047527+0000","last_active":"2024-05-27T08:36:57.176070+0000","last_peered":"2024-05-27T08:36:57.176070+0000","last_clean":"2024-05-27T08:36:57.176070+0000","last_became_active":"2024-05-27T08:35:07.047204+0000","last_became_peered":"2024-05-27T08:35:07.047204+0000","last_unstale":"2024-05-27T08:36:57.176070+0000","last_undegraded":"2024-05-27T08:36:57.176070+0000","last_fullsized":"2024-05-27T08:36:57.176070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:04:43.547847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375361+0000","last_change":"2024-05-27T08:35:07.049108+0000","last_active":"2024-05-27T08:35:58.375361+0000","last_peered":"2024-05-27T08:35:58.375361+0000","last_clean":"2024-05-27T08:35:58.375361+0000","last_became_active":"2024-05-27T08:35:07.048540+0000","last_became_peered":"2024-05-27T08:35:07.048540+0000","last_unstale":"2024-05-27T08:35:58.375361+0000","last_undegraded":"2024-05-27T08:35:58.375361+0000","last_fullsized":"2024-05-27T08:35:58.375361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:56.219362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367976+0000","last_change":"2024-05-27T08:35:07.049775+0000","last_active":"2024-05-27T08:35:58.367976+0000","last_peered":"2024-05-27T08:35:58.367976+0000","last_clean":"2024-05-27T08:35:58.367976+0000","last_became_active":"2024-05-27T08:35:07.049466+0000","last_became_peered":"2024-05-27T08:35:07.049466+0000","last_unstale":"2024-05-27T08:35:58.367976+0000","last_undegraded":"2024-05-27T08:35:58.367976+0000","last_fullsized":"2024-05-27T08:35:58.367976+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:18:30.394390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371274+0000","last_change":"2024-05-27T08:35:07.050870+0000","last_active":"2024-05-27T08:35:58.371274+0000","last_peered":"2024-05-27T08:35:58.371274+0000","last_clean":"2024-05-27T08:35:58.371274+0000","last_became_active":"2024-05-27T08:35:07.050527+0000","last_became_peered":"2024-05-27T08:35:07.050527+0000","last_unstale":"2024-05-27T08:35:58.371274+0000","last_undegraded":"2024-05-27T08:35:58.371274+0000","last_fullsized":"2024-05-27T08:35:58.371274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:56:41.477053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.172749+0000","last_change":"2024-05-27T08:35:07.052955+0000","last_active":"2024-05-27T08:36:57.172749+0000","last_peered":"2024-05-27T08:36:57.172749+0000","last_clean":"2024-05-27T08:36:57.172749+0000","last_became_active":"2024-05-27T08:35:07.052500+0000","last_became_peered":"2024-05-27T08:35:07.052500+0000","last_unstale":"2024-05-27T08:36:57.172749+0000","last_undegraded":"2024-05-27T08:36:57.172749+0000","last_fullsized":"2024-05-27T08:36:57.172749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:25.093408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.173029+0000","last_change":"2024-05-27T08:35:07.044866+0000","last_active":"2024-05-27T08:36:57.173029+0000","last_peered":"2024-05-27T08:36:57.173029+0000","last_clean":"2024-05-27T08:36:57.173029+0000","last_became_active":"2024-05-27T08:35:07.044554+0000","last_became_peered":"2024-05-27T08:35:07.044554+0000","last_unstale":"2024-05-27T08:36:57.173029+0000","last_undegraded":"2024-05-27T08:36:57.173029+0000","last_fullsized":"2024-05-27T08:36:57.173029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:45.624970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370530+0000","last_change":"2024-05-27T08:35:07.043985+0000","last_active":"2024-05-27T08:35:58.370530+0000","last_peered":"2024-05-27T08:35:58.370530+0000","last_clean":"2024-05-27T08:35:58.370530+0000","last_became_active":"2024-05-27T08:35:07.043694+0000","last_became_peered":"2024-05-27T08:35:07.043694+0000","last_unstale":"2024-05-27T08:35:58.370530+0000","last_undegraded":"2024-05-27T08:35:58.370530+0000","last_fullsized":"2024-05-27T08:35:58.370530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:22:49.796670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370414+0000","last_change":"2024-05-27T08:35:07.045542+0000","last_active":"2024-05-27T08:35:58.370414+0000","last_peered":"2024-05-27T08:35:58.370414+0000","last_clean":"2024-05-27T08:35:58.370414+0000","last_became_active":"2024-05-27T08:35:07.045209+0000","last_became_peered":"2024-05-27T08:35:07.045209+0000","last_unstale":"2024-05-27T08:35:58.370414+0000","last_undegraded":"2024-05-27T08:35:58.370414+0000","last_fullsized":"2024-05-27T08:35:58.370414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:57.733089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374546+0000","last_change":"2024-05-27T08:35:07.047193+0000","last_active":"2024-05-27T08:35:58.374546+0000","last_peered":"2024-05-27T08:35:58.374546+0000","last_clean":"2024-05-27T08:35:58.374546+0000","last_became_active":"2024-05-27T08:35:07.046862+0000","last_became_peered":"2024-05-27T08:35:07.046862+0000","last_unstale":"2024-05-27T08:35:58.374546+0000","last_undegraded":"2024-05-27T08:35:58.374546+0000","last_fullsized":"2024-05-27T08:35:58.374546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:21.288642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.188308+0000","last_change":"2024-05-27T08:35:07.045452+0000","last_active":"2024-05-27T08:36:57.188308+0000","last_peered":"2024-05-27T08:36:57.188308+0000","last_clean":"2024-05-27T08:36:57.188308+0000","last_became_active":"2024-05-27T08:35:07.044783+0000","last_became_peered":"2024-05-27T08:35:07.044783+0000","last_unstale":"2024-05-27T08:36:57.188308+0000","last_undegraded":"2024-05-27T08:36:57.188308+0000","last_fullsized":"2024-05-27T08:36:57.188308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:03.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.170797+0000","last_change":"2024-05-27T08:35:07.042864+0000","last_active":"2024-05-27T08:36:57.170797+0000","last_peered":"2024-05-27T08:36:57.170797+0000","last_clean":"2024-05-27T08:36:57.170797+0000","last_became_active":"2024-05-27T08:35:07.042523+0000","last_became_peered":"2024-05-27T08:35:07.042523+0000","last_unstale":"2024-05-27T08:36:57.170797+0000","last_undegraded":"2024-05-27T08:36:57.170797+0000","last_fullsized":"2024-05-27T08:36:57.170797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:34.275977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.369532+0000","last_change":"2024-05-27T08:35:07.042667+0000","last_active":"2024-05-27T08:35:58.369532+0000","last_peered":"2024-05-27T08:35:58.369532+0000","last_clean":"2024-05-27T08:35:58.369532+0000","last_became_active":"2024-05-27T08:35:07.042090+0000","last_became_peered":"2024-05-27T08:35:07.042090+0000","last_unstale":"2024-05-27T08:35:58.369532+0000","last_undegraded":"2024-05-27T08:35:58.369532+0000","last_fullsized":"2024-05-27T08:35:58.369532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:28.764641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"83'13","reported_seq":153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:56.450819+0000","last_change":"2024-05-27T08:35:58.391406+0000","last_active":"2024-05-27T08:36:56.450819+0000","last_peered":"2024-05-27T08:36:56.450819+0000","last_clean":"2024-05-27T08:36:56.450819+0000","last_became_active":"2024-05-27T08:35:58.376035+0000","last_became_peered":"2024-05-27T08:35:58.376035+0000","last_unstale":"2024-05-27T08:36:56.450819+0000","last_undegraded":"2024-05-27T08:36:56.450819+0000","last_fullsized":"2024-05-27T08:36:56.450819+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:54.751467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.172694+0000","last_change":"2024-05-27T08:35:58.376043+0000","last_active":"2024-05-27T08:36:57.172694+0000","last_peered":"2024-05-27T08:36:57.172694+0000","last_clean":"2024-05-27T08:36:57.172694+0000","last_became_active":"2024-05-27T08:35:58.375695+0000","last_became_peered":"2024-05-27T08:35:58.375695+0000","last_unstale":"2024-05-27T08:36:57.172694+0000","last_undegraded":"2024-05-27T08:36:57.172694+0000","last_fullsized":"2024-05-27T08:36:57.172694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:30:40.875555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368162+0000","last_change":"2024-05-27T08:35:04.245013+0000","last_active":"2024-05-27T08:35:58.368162+0000","last_peered":"2024-05-27T08:35:58.368162+0000","last_clean":"2024-05-27T08:35:58.368162+0000","last_became_active":"2024-05-27T08:35:04.244629+0000","last_became_peered":"2024-05-27T08:35:04.244629+0000","last_unstale":"2024-05-27T08:35:58.368162+0000","last_undegraded":"2024-05-27T08:35:58.368162+0000","last_fullsized":"2024-05-27T08:35:58.368162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:41:43.747592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371029+0000","last_change":"2024-05-27T08:35:07.041776+0000","last_active":"2024-05-27T08:35:58.371029+0000","last_peered":"2024-05-27T08:35:58.371029+0000","last_clean":"2024-05-27T08:35:58.371029+0000","last_became_active":"2024-05-27T08:35:07.041400+0000","last_became_peered":"2024-05-27T08:35:07.041400+0000","last_unstale":"2024-05-27T08:35:58.371029+0000","last_undegraded":"2024-05-27T08:35:58.371029+0000","last_fullsized":"2024-05-27T08:35:58.371029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:10.091310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368616+0000","last_change":"2024-05-27T08:35:04.023072+0000","last_active":"2024-05-27T08:35:58.368616+0000","last_peered":"2024-05-27T08:35:58.368616+0000","last_clean":"2024-05-27T08:35:58.368616+0000","last_became_active":"2024-05-27T08:35:04.022232+0000","last_became_peered":"2024-05-27T08:35:04.022232+0000","last_unstale":"2024-05-27T08:35:58.368616+0000","last_undegraded":"2024-05-27T08:35:58.368616+0000","last_fullsized":"2024-05-27T08:35:58.368616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:49:57.240621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374796+0000","last_change":"2024-05-27T08:35:07.044727+0000","last_active":"2024-05-27T08:35:58.374796+0000","last_peered":"2024-05-27T08:35:58.374796+0000","last_clean":"2024-05-27T08:35:58.374796+0000","last_became_active":"2024-05-27T08:35:07.044352+0000","last_became_peered":"2024-05-27T08:35:07.044352+0000","last_unstale":"2024-05-27T08:35:58.374796+0000","last_undegraded":"2024-05-27T08:35:58.374796+0000","last_fullsized":"2024-05-27T08:35:58.374796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:52:43.473606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.382377+0000","last_change":"2024-05-27T08:35:58.382377+0000","last_active":"2024-05-27T08:35:58.382377+0000","last_peered":"2024-05-27T08:35:58.382377+0000","last_clean":"2024-05-27T08:35:58.382377+0000","last_became_active":"2024-05-27T08:35:58.376093+0000","last_became_peered":"2024-05-27T08:35:58.376093+0000","last_unstale":"2024-05-27T08:35:58.382377+0000","last_undegraded":"2024-05-27T08:35:58.382377+0000","last_fullsized":"2024-05-27T08:35:58.382377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:33.792015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.180144+0000","last_change":"2024-05-27T08:35:07.055562+0000","last_active":"2024-05-27T08:36:57.180144+0000","last_peered":"2024-05-27T08:36:57.180144+0000","last_clean":"2024-05-27T08:36:57.180144+0000","last_became_active":"2024-05-27T08:35:07.055209+0000","last_became_peered":"2024-05-27T08:35:07.055209+0000","last_unstale":"2024-05-27T08:36:57.180144+0000","last_undegraded":"2024-05-27T08:36:57.180144+0000","last_fullsized":"2024-05-27T08:36:57.180144+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:04.920534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'7","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.411237+0000","last_change":"2024-05-27T08:35:04.022778+0000","last_active":"2024-05-27T08:36:57.411237+0000","last_peered":"2024-05-27T08:36:57.411237+0000","last_clean":"2024-05-27T08:36:57.411237+0000","last_became_active":"2024-05-27T08:35:04.022311+0000","last_became_peered":"2024-05-27T08:35:04.022311+0000","last_unstale":"2024-05-27T08:36:57.411237+0000","last_undegraded":"2024-05-27T08:36:57.411237+0000","last_fullsized":"2024-05-27T08:36:57.411237+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:48.174774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.168983+0000","last_change":"2024-05-27T08:35:07.040182+0000","last_active":"2024-05-27T08:36:57.168983+0000","last_peered":"2024-05-27T08:36:57.168983+0000","last_clean":"2024-05-27T08:36:57.168983+0000","last_became_active":"2024-05-27T08:35:07.039800+0000","last_became_peered":"2024-05-27T08:35:07.039800+0000","last_unstale":"2024-05-27T08:36:57.168983+0000","last_undegraded":"2024-05-27T08:36:57.168983+0000","last_fullsized":"2024-05-27T08:36:57.168983+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:53:53.470284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367626+0000","last_change":"2024-05-27T08:35:04.023757+0000","last_active":"2024-05-27T08:35:58.367626+0000","last_peered":"2024-05-27T08:35:58.367626+0000","last_clean":"2024-05-27T08:35:58.367626+0000","last_became_active":"2024-05-27T08:35:04.023428+0000","last_became_peered":"2024-05-27T08:35:04.023428+0000","last_unstale":"2024-05-27T08:35:58.367626+0000","last_undegraded":"2024-05-27T08:35:58.367626+0000","last_fullsized":"2024-05-27T08:35:58.367626+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:45.669006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.178370+0000","last_change":"2024-05-27T08:35:58.407697+0000","last_active":"2024-05-27T08:36:57.178370+0000","last_peered":"2024-05-27T08:36:57.178370+0000","last_clean":"2024-05-27T08:36:57.178370+0000","last_became_active":"2024-05-27T08:35:58.407379+0000","last_became_peered":"2024-05-27T08:35:58.407379+0000","last_unstale":"2024-05-27T08:36:57.178370+0000","last_undegraded":"2024-05-27T08:36:57.178370+0000","last_fullsized":"2024-05-27T08:36:57.178370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:06.534914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364641+0000","last_change":"2024-05-27T08:35:04.015581+0000","last_active":"2024-05-27T08:35:58.364641+0000","last_peered":"2024-05-27T08:35:58.364641+0000","last_clean":"2024-05-27T08:35:58.364641+0000","last_became_active":"2024-05-27T08:35:04.015215+0000","last_became_peered":"2024-05-27T08:35:04.015215+0000","last_unstale":"2024-05-27T08:35:58.364641+0000","last_undegraded":"2024-05-27T08:35:58.364641+0000","last_fullsized":"2024-05-27T08:35:58.364641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:21:51.942008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.177506+0000","last_change":"2024-05-27T08:35:07.049798+0000","last_active":"2024-05-27T08:36:57.177506+0000","last_peered":"2024-05-27T08:36:57.177506+0000","last_clean":"2024-05-27T08:36:57.177506+0000","last_became_active":"2024-05-27T08:35:07.048686+0000","last_became_peered":"2024-05-27T08:35:07.048686+0000","last_unstale":"2024-05-27T08:36:57.177506+0000","last_undegraded":"2024-05-27T08:36:57.177506+0000","last_fullsized":"2024-05-27T08:36:57.177506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:39:23.817581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374818+0000","last_change":"2024-05-27T08:35:04.023826+0000","last_active":"2024-05-27T08:35:58.374818+0000","last_peered":"2024-05-27T08:35:58.374818+0000","last_clean":"2024-05-27T08:35:58.374818+0000","last_became_active":"2024-05-27T08:35:04.023410+0000","last_became_peered":"2024-05-27T08:35:04.023410+0000","last_unstale":"2024-05-27T08:35:58.374818+0000","last_undegraded":"2024-05-27T08:35:58.374818+0000","last_fullsized":"2024-05-27T08:35:58.374818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:38:47.855186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.178960+0000","last_change":"2024-05-27T08:35:07.038810+0000","last_active":"2024-05-27T08:36:57.178960+0000","last_peered":"2024-05-27T08:36:57.178960+0000","last_clean":"2024-05-27T08:36:57.178960+0000","last_became_active":"2024-05-27T08:35:07.038480+0000","last_became_peered":"2024-05-27T08:35:07.038480+0000","last_unstale":"2024-05-27T08:36:57.178960+0000","last_undegraded":"2024-05-27T08:36:57.178960+0000","last_fullsized":"2024-05-27T08:36:57.178960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:14.429081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":212,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:54.405504+0000","last_change":"2024-05-27T08:35:04.026934+0000","last_active":"2024-05-27T08:36:54.405504+0000","last_peered":"2024-05-27T08:36:54.405504+0000","last_clean":"2024-05-27T08:36:54.405504+0000","last_became_active":"2024-05-27T08:35:04.026577+0000","last_became_peered":"2024-05-27T08:35:04.026577+0000","last_unstale":"2024-05-27T08:36:54.405504+0000","last_undegraded":"2024-05-27T08:36:54.405504+0000","last_fullsized":"2024-05-27T08:36:54.405504+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:59:56.067856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":182,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375300+0000","last_change":"2024-05-27T08:35:07.043840+0000","last_active":"2024-05-27T08:35:58.375300+0000","last_peered":"2024-05-27T08:35:58.375300+0000","last_clean":"2024-05-27T08:35:58.375300+0000","last_became_active":"2024-05-27T08:35:07.043224+0000","last_became_peered":"2024-05-27T08:35:07.043224+0000","last_unstale":"2024-05-27T08:35:58.375300+0000","last_undegraded":"2024-05-27T08:35:58.375300+0000","last_fullsized":"2024-05-27T08:35:58.375300+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:20.212038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371395+0000","last_change":"2024-05-27T08:35:04.022602+0000","last_active":"2024-05-27T08:35:58.371395+0000","last_peered":"2024-05-27T08:35:58.371395+0000","last_clean":"2024-05-27T08:35:58.371395+0000","last_became_active":"2024-05-27T08:35:04.022203+0000","last_became_peered":"2024-05-27T08:35:04.022203+0000","last_unstale":"2024-05-27T08:35:58.371395+0000","last_undegraded":"2024-05-27T08:35:58.371395+0000","last_fullsized":"2024-05-27T08:35:58.371395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:51.490696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370927+0000","last_change":"2024-05-27T08:35:07.047257+0000","last_active":"2024-05-27T08:35:58.370927+0000","last_peered":"2024-05-27T08:35:58.370927+0000","last_clean":"2024-05-27T08:35:58.370927+0000","last_became_active":"2024-05-27T08:35:07.046639+0000","last_became_peered":"2024-05-27T08:35:07.046639+0000","last_unstale":"2024-05-27T08:35:58.370927+0000","last_undegraded":"2024-05-27T08:35:58.370927+0000","last_fullsized":"2024-05-27T08:35:58.370927+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:04.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.376394+0000","last_change":"2024-05-27T08:35:04.245295+0000","last_active":"2024-05-27T08:35:58.376394+0000","last_peered":"2024-05-27T08:35:58.376394+0000","last_clean":"2024-05-27T08:35:58.376394+0000","last_became_active":"2024-05-27T08:35:04.244927+0000","last_became_peered":"2024-05-27T08:35:04.244927+0000","last_unstale":"2024-05-27T08:35:58.376394+0000","last_undegraded":"2024-05-27T08:35:58.376394+0000","last_fullsized":"2024-05-27T08:35:58.376394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:58.893480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374886+0000","last_change":"2024-05-27T08:35:07.048872+0000","last_active":"2024-05-27T08:35:58.374886+0000","last_peered":"2024-05-27T08:35:58.374886+0000","last_clean":"2024-05-27T08:35:58.374886+0000","last_became_active":"2024-05-27T08:35:07.048488+0000","last_became_peered":"2024-05-27T08:35:07.048488+0000","last_unstale":"2024-05-27T08:35:58.374886+0000","last_undegraded":"2024-05-27T08:35:58.374886+0000","last_fullsized":"2024-05-27T08:35:58.374886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:31.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":248,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:54.405361+0000","last_change":"2024-05-27T08:35:04.024718+0000","last_active":"2024-05-27T08:36:54.405361+0000","last_peered":"2024-05-27T08:36:54.405361+0000","last_clean":"2024-05-27T08:36:54.405361+0000","last_became_active":"2024-05-27T08:35:04.024155+0000","last_became_peered":"2024-05-27T08:35:04.024155+0000","last_unstale":"2024-05-27T08:36:54.405361+0000","last_undegraded":"2024-05-27T08:36:54.405361+0000","last_fullsized":"2024-05-27T08:36:54.405361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:22.573337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":191,"num_read_kb":153,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.179061+0000","last_change":"2024-05-27T08:35:07.051342+0000","last_active":"2024-05-27T08:36:57.179061+0000","last_peered":"2024-05-27T08:36:57.179061+0000","last_clean":"2024-05-27T08:36:57.179061+0000","last_became_active":"2024-05-27T08:35:07.051030+0000","last_became_peered":"2024-05-27T08:35:07.051030+0000","last_unstale":"2024-05-27T08:36:57.179061+0000","last_undegraded":"2024-05-27T08:36:57.179061+0000","last_fullsized":"2024-05-27T08:36:57.179061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:48.245036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368633+0000","last_change":"2024-05-27T08:35:04.023717+0000","last_active":"2024-05-27T08:35:58.368633+0000","last_peered":"2024-05-27T08:35:58.368633+0000","last_clean":"2024-05-27T08:35:58.368633+0000","last_became_active":"2024-05-27T08:35:04.022017+0000","last_became_peered":"2024-05-27T08:35:04.022017+0000","last_unstale":"2024-05-27T08:35:58.368633+0000","last_undegraded":"2024-05-27T08:35:58.368633+0000","last_fullsized":"2024-05-27T08:35:58.368633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:55.834307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.180299+0000","last_change":"2024-05-27T08:35:07.043980+0000","last_active":"2024-05-27T08:36:57.180299+0000","last_peered":"2024-05-27T08:36:57.180299+0000","last_clean":"2024-05-27T08:36:57.180299+0000","last_became_active":"2024-05-27T08:35:07.043672+0000","last_became_peered":"2024-05-27T08:35:07.043672+0000","last_unstale":"2024-05-27T08:36:57.180299+0000","last_undegraded":"2024-05-27T08:36:57.180299+0000","last_fullsized":"2024-05-27T08:36:57.180299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:03:30.243966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.375453+0000","last_change":"2024-05-27T08:35:04.022731+0000","last_active":"2024-05-27T08:35:58.375453+0000","last_peered":"2024-05-27T08:35:58.375453+0000","last_clean":"2024-05-27T08:35:58.375453+0000","last_became_active":"2024-05-27T08:35:04.022349+0000","last_became_peered":"2024-05-27T08:35:04.022349+0000","last_unstale":"2024-05-27T08:35:58.375453+0000","last_undegraded":"2024-05-27T08:35:58.375453+0000","last_fullsized":"2024-05-27T08:35:58.375453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:38.651520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371218+0000","last_change":"2024-05-27T08:35:07.052623+0000","last_active":"2024-05-27T08:35:58.371218+0000","last_peered":"2024-05-27T08:35:58.371218+0000","last_clean":"2024-05-27T08:35:58.371218+0000","last_became_active":"2024-05-27T08:35:07.052313+0000","last_became_peered":"2024-05-27T08:35:07.052313+0000","last_unstale":"2024-05-27T08:35:58.371218+0000","last_undegraded":"2024-05-27T08:35:58.371218+0000","last_fullsized":"2024-05-27T08:35:58.371218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:24.612523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'7","reported_seq":131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.411296+0000","last_change":"2024-05-27T08:35:04.023756+0000","last_active":"2024-05-27T08:36:57.411296+0000","last_peered":"2024-05-27T08:36:57.411296+0000","last_clean":"2024-05-27T08:36:57.411296+0000","last_became_active":"2024-05-27T08:35:04.023401+0000","last_became_peered":"2024-05-27T08:35:04.023401+0000","last_unstale":"2024-05-27T08:36:57.411296+0000","last_undegraded":"2024-05-27T08:36:57.411296+0000","last_fullsized":"2024-05-27T08:36:57.411296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:34.887246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370820+0000","last_change":"2024-05-27T08:35:07.050720+0000","last_active":"2024-05-27T08:35:58.370820+0000","last_peered":"2024-05-27T08:35:58.370820+0000","last_clean":"2024-05-27T08:35:58.370820+0000","last_became_active":"2024-05-27T08:35:07.050135+0000","last_became_peered":"2024-05-27T08:35:07.050135+0000","last_unstale":"2024-05-27T08:35:58.370820+0000","last_undegraded":"2024-05-27T08:35:58.370820+0000","last_fullsized":"2024-05-27T08:35:58.370820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:40.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.421002+0000","last_change":"2024-05-27T08:35:58.421002+0000","last_active":"2024-05-27T08:35:58.421002+0000","last_peered":"2024-05-27T08:35:58.421002+0000","last_clean":"2024-05-27T08:35:58.421002+0000","last_became_active":"2024-05-27T08:35:58.408889+0000","last_became_peered":"2024-05-27T08:35:58.408889+0000","last_unstale":"2024-05-27T08:35:58.421002+0000","last_undegraded":"2024-05-27T08:35:58.421002+0000","last_fullsized":"2024-05-27T08:35:58.421002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:29.705436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.160637+0000","last_change":"2024-05-27T08:35:07.050504+0000","last_active":"2024-05-27T08:36:57.160637+0000","last_peered":"2024-05-27T08:36:57.160637+0000","last_clean":"2024-05-27T08:36:57.160637+0000","last_became_active":"2024-05-27T08:35:07.049368+0000","last_became_peered":"2024-05-27T08:35:07.049368+0000","last_unstale":"2024-05-27T08:36:57.160637+0000","last_undegraded":"2024-05-27T08:36:57.160637+0000","last_fullsized":"2024-05-27T08:36:57.160637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:52.996313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368599+0000","last_change":"2024-05-27T08:35:04.023924+0000","last_active":"2024-05-27T08:35:58.368599+0000","last_peered":"2024-05-27T08:35:58.368599+0000","last_clean":"2024-05-27T08:35:58.368599+0000","last_became_active":"2024-05-27T08:35:04.023548+0000","last_became_peered":"2024-05-27T08:35:04.023548+0000","last_unstale":"2024-05-27T08:35:58.368599+0000","last_undegraded":"2024-05-27T08:35:58.368599+0000","last_fullsized":"2024-05-27T08:35:58.368599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:50.441490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.170772+0000","last_change":"2024-05-27T08:35:58.376335+0000","last_active":"2024-05-27T08:36:57.170772+0000","last_peered":"2024-05-27T08:36:57.170772+0000","last_clean":"2024-05-27T08:36:57.170772+0000","last_became_active":"2024-05-27T08:35:58.375926+0000","last_became_peered":"2024-05-27T08:35:58.375926+0000","last_unstale":"2024-05-27T08:36:57.170772+0000","last_undegraded":"2024-05-27T08:36:57.170772+0000","last_fullsized":"2024-05-27T08:36:57.170772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:33.892172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374310+0000","last_change":"2024-05-27T08:35:58.374310+0000","last_active":"2024-05-27T08:35:58.374310+0000","last_peered":"2024-05-27T08:35:58.374310+0000","last_clean":"2024-05-27T08:35:58.374310+0000","last_became_active":"2024-05-27T08:35:58.373929+0000","last_became_peered":"2024-05-27T08:35:58.373929+0000","last_unstale":"2024-05-27T08:35:58.374310+0000","last_undegraded":"2024-05-27T08:35:58.374310+0000","last_fullsized":"2024-05-27T08:35:58.374310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:20.901125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'1","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367829+0000","last_change":"2024-05-27T08:35:07.039182+0000","last_active":"2024-05-27T08:35:58.367829+0000","last_peered":"2024-05-27T08:35:58.367829+0000","last_clean":"2024-05-27T08:35:58.367829+0000","last_became_active":"2024-05-27T08:35:07.038793+0000","last_became_peered":"2024-05-27T08:35:07.038793+0000","last_unstale":"2024-05-27T08:35:58.367829+0000","last_undegraded":"2024-05-27T08:35:58.367829+0000","last_fullsized":"2024-05-27T08:35:58.367829+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:03.839681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"83'5","reported_seq":124,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:55.450060+0000","last_change":"2024-05-27T08:35:04.023194+0000","last_active":"2024-05-27T08:36:55.450060+0000","last_peered":"2024-05-27T08:36:55.450060+0000","last_clean":"2024-05-27T08:36:55.450060+0000","last_became_active":"2024-05-27T08:35:04.022823+0000","last_became_peered":"2024-05-27T08:35:04.022823+0000","last_unstale":"2024-05-27T08:36:55.450060+0000","last_undegraded":"2024-05-27T08:36:55.450060+0000","last_fullsized":"2024-05-27T08:36:55.450060+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:44:28.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367676+0000","last_change":"2024-05-27T08:35:07.050717+0000","last_active":"2024-05-27T08:35:58.367676+0000","last_peered":"2024-05-27T08:35:58.367676+0000","last_clean":"2024-05-27T08:35:58.367676+0000","last_became_active":"2024-05-27T08:35:07.050413+0000","last_became_peered":"2024-05-27T08:35:07.050413+0000","last_unstale":"2024-05-27T08:35:58.367676+0000","last_undegraded":"2024-05-27T08:35:58.367676+0000","last_fullsized":"2024-05-27T08:35:58.367676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:00:27.693141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.169850+0000","last_change":"2024-05-27T08:35:07.053614+0000","last_active":"2024-05-27T08:36:57.169850+0000","last_peered":"2024-05-27T08:36:57.169850+0000","last_clean":"2024-05-27T08:36:57.169850+0000","last_became_active":"2024-05-27T08:35:07.053253+0000","last_became_peered":"2024-05-27T08:35:07.053253+0000","last_unstale":"2024-05-27T08:36:57.169850+0000","last_undegraded":"2024-05-27T08:36:57.169850+0000","last_fullsized":"2024-05-27T08:36:57.169850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:15:08.985984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":127,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.135451+0000","last_change":"2024-05-27T08:35:04.246500+0000","last_active":"2024-05-27T08:36:57.135451+0000","last_peered":"2024-05-27T08:36:57.135451+0000","last_clean":"2024-05-27T08:36:57.135451+0000","last_became_active":"2024-05-27T08:35:04.244575+0000","last_became_peered":"2024-05-27T08:35:04.244575+0000","last_unstale":"2024-05-27T08:36:57.135451+0000","last_undegraded":"2024-05-27T08:36:57.135451+0000","last_fullsized":"2024-05-27T08:36:57.135451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:24:10.818057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.186289+0000","last_change":"2024-05-27T08:35:07.048488+0000","last_active":"2024-05-27T08:36:57.186289+0000","last_peered":"2024-05-27T08:36:57.186289+0000","last_clean":"2024-05-27T08:36:57.186289+0000","last_became_active":"2024-05-27T08:35:07.048172+0000","last_became_peered":"2024-05-27T08:35:07.048172+0000","last_unstale":"2024-05-27T08:36:57.186289+0000","last_undegraded":"2024-05-27T08:36:57.186289+0000","last_fullsized":"2024-05-27T08:36:57.186289+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:01.632944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.407485+0000","last_change":"2024-05-27T08:35:04.022807+0000","last_active":"2024-05-27T08:35:58.407485+0000","last_peered":"2024-05-27T08:35:58.407485+0000","last_clean":"2024-05-27T08:35:58.407485+0000","last_became_active":"2024-05-27T08:35:04.021198+0000","last_became_peered":"2024-05-27T08:35:04.021198+0000","last_unstale":"2024-05-27T08:35:58.407485+0000","last_undegraded":"2024-05-27T08:35:58.407485+0000","last_fullsized":"2024-05-27T08:35:58.407485+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:42:10.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":101,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370999+0000","last_change":"2024-05-27T08:33:45.683600+0000","last_active":"2024-05-27T08:35:58.370999+0000","last_peered":"2024-05-27T08:35:58.370999+0000","last_clean":"2024-05-27T08:35:58.370999+0000","last_became_active":"2024-05-27T08:33:45.677989+0000","last_became_peered":"2024-05-27T08:33:45.677989+0000","last_unstale":"2024-05-27T08:35:58.370999+0000","last_undegraded":"2024-05-27T08:35:58.370999+0000","last_fullsized":"2024-05-27T08:35:58.370999+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_clean_scrub_stamp":"2024-05-27T08:30:25.507727+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:48:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.174167+0000","last_change":"2024-05-27T08:35:07.045879+0000","last_active":"2024-05-27T08:36:57.174167+0000","last_peered":"2024-05-27T08:36:57.174167+0000","last_clean":"2024-05-27T08:36:57.174167+0000","last_became_active":"2024-05-27T08:35:07.045522+0000","last_became_peered":"2024-05-27T08:35:07.045522+0000","last_unstale":"2024-05-27T08:36:57.174167+0000","last_undegraded":"2024-05-27T08:36:57.174167+0000","last_fullsized":"2024-05-27T08:36:57.174167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:30.393769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371043+0000","last_change":"2024-05-27T08:35:04.022419+0000","last_active":"2024-05-27T08:35:58.371043+0000","last_peered":"2024-05-27T08:35:58.371043+0000","last_clean":"2024-05-27T08:35:58.371043+0000","last_became_active":"2024-05-27T08:35:04.022039+0000","last_became_peered":"2024-05-27T08:35:04.022039+0000","last_unstale":"2024-05-27T08:35:58.371043+0000","last_undegraded":"2024-05-27T08:35:58.371043+0000","last_fullsized":"2024-05-27T08:35:58.371043+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:31.260596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368426+0000","last_change":"2024-05-27T08:35:04.245214+0000","last_active":"2024-05-27T08:35:58.368426+0000","last_peered":"2024-05-27T08:35:58.368426+0000","last_clean":"2024-05-27T08:35:58.368426+0000","last_became_active":"2024-05-27T08:35:04.244768+0000","last_became_peered":"2024-05-27T08:35:04.244768+0000","last_unstale":"2024-05-27T08:35:58.368426+0000","last_undegraded":"2024-05-27T08:35:58.368426+0000","last_fullsized":"2024-05-27T08:35:58.368426+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:34.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.171872+0000","last_change":"2024-05-27T08:35:07.042721+0000","last_active":"2024-05-27T08:36:57.171872+0000","last_peered":"2024-05-27T08:36:57.171872+0000","last_clean":"2024-05-27T08:36:57.171872+0000","last_became_active":"2024-05-27T08:35:07.042435+0000","last_became_peered":"2024-05-27T08:35:07.042435+0000","last_unstale":"2024-05-27T08:36:57.171872+0000","last_undegraded":"2024-05-27T08:36:57.171872+0000","last_fullsized":"2024-05-27T08:36:57.171872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:49.186753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368668+0000","last_change":"2024-05-27T08:35:04.017995+0000","last_active":"2024-05-27T08:35:58.368668+0000","last_peered":"2024-05-27T08:35:58.368668+0000","last_clean":"2024-05-27T08:35:58.368668+0000","last_became_active":"2024-05-27T08:35:04.017675+0000","last_became_peered":"2024-05-27T08:35:04.017675+0000","last_unstale":"2024-05-27T08:35:58.368668+0000","last_undegraded":"2024-05-27T08:35:58.368668+0000","last_fullsized":"2024-05-27T08:35:58.368668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:23:15.452624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.180121+0000","last_change":"2024-05-27T08:35:07.046553+0000","last_active":"2024-05-27T08:36:57.180121+0000","last_peered":"2024-05-27T08:36:57.180121+0000","last_clean":"2024-05-27T08:36:57.180121+0000","last_became_active":"2024-05-27T08:35:07.046229+0000","last_became_peered":"2024-05-27T08:35:07.046229+0000","last_unstale":"2024-05-27T08:36:57.180121+0000","last_undegraded":"2024-05-27T08:36:57.180121+0000","last_fullsized":"2024-05-27T08:36:57.180121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:21.024307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.407482+0000","last_change":"2024-05-27T08:35:04.262439+0000","last_active":"2024-05-27T08:35:58.407482+0000","last_peered":"2024-05-27T08:35:58.407482+0000","last_clean":"2024-05-27T08:35:58.407482+0000","last_became_active":"2024-05-27T08:35:04.261958+0000","last_became_peered":"2024-05-27T08:35:04.261958+0000","last_unstale":"2024-05-27T08:35:58.407482+0000","last_undegraded":"2024-05-27T08:35:58.407482+0000","last_fullsized":"2024-05-27T08:35:58.407482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:11:05.763068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370773+0000","last_change":"2024-05-27T08:35:07.042307+0000","last_active":"2024-05-27T08:35:58.370773+0000","last_peered":"2024-05-27T08:35:58.370773+0000","last_clean":"2024-05-27T08:35:58.370773+0000","last_became_active":"2024-05-27T08:35:07.042001+0000","last_became_peered":"2024-05-27T08:35:07.042001+0000","last_unstale":"2024-05-27T08:35:58.370773+0000","last_undegraded":"2024-05-27T08:35:58.370773+0000","last_fullsized":"2024-05-27T08:35:58.370773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:51.030157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374562+0000","last_change":"2024-05-27T08:35:04.245436+0000","last_active":"2024-05-27T08:35:58.374562+0000","last_peered":"2024-05-27T08:35:58.374562+0000","last_clean":"2024-05-27T08:35:58.374562+0000","last_became_active":"2024-05-27T08:35:04.245039+0000","last_became_peered":"2024-05-27T08:35:04.245039+0000","last_unstale":"2024-05-27T08:35:58.374562+0000","last_undegraded":"2024-05-27T08:35:58.374562+0000","last_fullsized":"2024-05-27T08:35:58.374562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:10:11.756864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.150724+0000","last_change":"2024-05-27T08:35:07.051524+0000","last_active":"2024-05-27T08:36:57.150724+0000","last_peered":"2024-05-27T08:36:57.150724+0000","last_clean":"2024-05-27T08:36:57.150724+0000","last_became_active":"2024-05-27T08:35:07.050676+0000","last_became_peered":"2024-05-27T08:35:07.050676+0000","last_unstale":"2024-05-27T08:36:57.150724+0000","last_undegraded":"2024-05-27T08:36:57.150724+0000","last_fullsized":"2024-05-27T08:36:57.150724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:06.818977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.367648+0000","last_change":"2024-05-27T08:35:04.022064+0000","last_active":"2024-05-27T08:35:58.367648+0000","last_peered":"2024-05-27T08:35:58.367648+0000","last_clean":"2024-05-27T08:35:58.367648+0000","last_became_active":"2024-05-27T08:35:04.021691+0000","last_became_peered":"2024-05-27T08:35:04.021691+0000","last_unstale":"2024-05-27T08:35:58.367648+0000","last_undegraded":"2024-05-27T08:35:58.367648+0000","last_fullsized":"2024-05-27T08:35:58.367648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:54.557756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.181431+0000","last_change":"2024-05-27T08:35:07.049666+0000","last_active":"2024-05-27T08:36:57.181431+0000","last_peered":"2024-05-27T08:36:57.181431+0000","last_clean":"2024-05-27T08:36:57.181431+0000","last_became_active":"2024-05-27T08:35:07.049312+0000","last_became_peered":"2024-05-27T08:35:07.049312+0000","last_unstale":"2024-05-27T08:36:57.181431+0000","last_undegraded":"2024-05-27T08:36:57.181431+0000","last_fullsized":"2024-05-27T08:36:57.181431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:37:53.704582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368318+0000","last_change":"2024-05-27T08:35:04.018457+0000","last_active":"2024-05-27T08:35:58.368318+0000","last_peered":"2024-05-27T08:35:58.368318+0000","last_clean":"2024-05-27T08:35:58.368318+0000","last_became_active":"2024-05-27T08:35:04.018008+0000","last_became_peered":"2024-05-27T08:35:04.018008+0000","last_unstale":"2024-05-27T08:35:58.368318+0000","last_undegraded":"2024-05-27T08:35:58.368318+0000","last_fullsized":"2024-05-27T08:35:58.368318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:08:13.228725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.180811+0000","last_change":"2024-05-27T08:35:07.047040+0000","last_active":"2024-05-27T08:36:57.180811+0000","last_peered":"2024-05-27T08:36:57.180811+0000","last_clean":"2024-05-27T08:36:57.180811+0000","last_became_active":"2024-05-27T08:35:07.046731+0000","last_became_peered":"2024-05-27T08:35:07.046731+0000","last_unstale":"2024-05-27T08:36:57.180811+0000","last_undegraded":"2024-05-27T08:36:57.180811+0000","last_fullsized":"2024-05-27T08:36:57.180811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:25.368349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368147+0000","last_change":"2024-05-27T08:35:04.022804+0000","last_active":"2024-05-27T08:35:58.368147+0000","last_peered":"2024-05-27T08:35:58.368147+0000","last_clean":"2024-05-27T08:35:58.368147+0000","last_became_active":"2024-05-27T08:35:04.022458+0000","last_became_peered":"2024-05-27T08:35:04.022458+0000","last_unstale":"2024-05-27T08:35:58.368147+0000","last_undegraded":"2024-05-27T08:35:58.368147+0000","last_fullsized":"2024-05-27T08:35:58.368147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:01.029226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371185+0000","last_change":"2024-05-27T08:35:07.049136+0000","last_active":"2024-05-27T08:35:58.371185+0000","last_peered":"2024-05-27T08:35:58.371185+0000","last_clean":"2024-05-27T08:35:58.371185+0000","last_became_active":"2024-05-27T08:35:07.048793+0000","last_became_peered":"2024-05-27T08:35:07.048793+0000","last_unstale":"2024-05-27T08:35:58.371185+0000","last_undegraded":"2024-05-27T08:35:58.371185+0000","last_fullsized":"2024-05-27T08:35:58.371185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:10:21.191743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'5","reported_seq":122,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:52.403534+0000","last_change":"2024-05-27T08:35:04.262102+0000","last_active":"2024-05-27T08:36:52.403534+0000","last_peered":"2024-05-27T08:36:52.403534+0000","last_clean":"2024-05-27T08:36:52.403534+0000","last_became_active":"2024-05-27T08:35:04.261665+0000","last_became_peered":"2024-05-27T08:35:04.261665+0000","last_unstale":"2024-05-27T08:36:52.403534+0000","last_undegraded":"2024-05-27T08:36:52.403534+0000","last_fullsized":"2024-05-27T08:36:52.403534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:57:57.712197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":92,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371431+0000","last_change":"2024-05-27T08:35:07.051557+0000","last_active":"2024-05-27T08:35:58.371431+0000","last_peered":"2024-05-27T08:35:58.371431+0000","last_clean":"2024-05-27T08:35:58.371431+0000","last_became_active":"2024-05-27T08:35:07.051246+0000","last_became_peered":"2024-05-27T08:35:07.051246+0000","last_unstale":"2024-05-27T08:35:58.371431+0000","last_undegraded":"2024-05-27T08:35:58.371431+0000","last_fullsized":"2024-05-27T08:35:58.371431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:08:13.248301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'3","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.376106+0000","last_change":"2024-05-27T08:35:04.022328+0000","last_active":"2024-05-27T08:35:58.376106+0000","last_peered":"2024-05-27T08:35:58.376106+0000","last_clean":"2024-05-27T08:35:58.376106+0000","last_became_active":"2024-05-27T08:35:04.021711+0000","last_became_peered":"2024-05-27T08:35:04.021711+0000","last_unstale":"2024-05-27T08:35:58.376106+0000","last_undegraded":"2024-05-27T08:35:58.376106+0000","last_fullsized":"2024-05-27T08:35:58.376106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:32.841474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371211+0000","last_change":"2024-05-27T08:35:07.056768+0000","last_active":"2024-05-27T08:35:58.371211+0000","last_peered":"2024-05-27T08:35:58.371211+0000","last_clean":"2024-05-27T08:35:58.371211+0000","last_became_active":"2024-05-27T08:35:07.056482+0000","last_became_peered":"2024-05-27T08:35:07.056482+0000","last_unstale":"2024-05-27T08:35:58.371211+0000","last_undegraded":"2024-05-27T08:35:58.371211+0000","last_fullsized":"2024-05-27T08:35:58.371211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:23.515625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.364744+0000","last_change":"2024-05-27T08:35:04.016613+0000","last_active":"2024-05-27T08:35:58.364744+0000","last_peered":"2024-05-27T08:35:58.364744+0000","last_clean":"2024-05-27T08:35:58.364744+0000","last_became_active":"2024-05-27T08:35:04.016268+0000","last_became_peered":"2024-05-27T08:35:04.016268+0000","last_unstale":"2024-05-27T08:35:58.364744+0000","last_undegraded":"2024-05-27T08:35:58.364744+0000","last_fullsized":"2024-05-27T08:35:58.364744+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:11.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.371530+0000","last_change":"2024-05-27T08:35:07.051515+0000","last_active":"2024-05-27T08:35:58.371530+0000","last_peered":"2024-05-27T08:35:58.371530+0000","last_clean":"2024-05-27T08:35:58.371530+0000","last_became_active":"2024-05-27T08:35:07.051197+0000","last_became_peered":"2024-05-27T08:35:07.051197+0000","last_unstale":"2024-05-27T08:35:58.371530+0000","last_undegraded":"2024-05-27T08:35:58.371530+0000","last_fullsized":"2024-05-27T08:35:58.371530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:49.233474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.173780+0000","last_change":"2024-05-27T08:35:07.051960+0000","last_active":"2024-05-27T08:36:57.173780+0000","last_peered":"2024-05-27T08:36:57.173780+0000","last_clean":"2024-05-27T08:36:57.173780+0000","last_became_active":"2024-05-27T08:35:07.051395+0000","last_became_peered":"2024-05-27T08:35:07.051395+0000","last_unstale":"2024-05-27T08:36:57.173780+0000","last_undegraded":"2024-05-27T08:36:57.173780+0000","last_fullsized":"2024-05-27T08:36:57.173780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:57.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368861+0000","last_change":"2024-05-27T08:35:07.050096+0000","last_active":"2024-05-27T08:35:58.368861+0000","last_peered":"2024-05-27T08:35:58.368861+0000","last_clean":"2024-05-27T08:35:58.368861+0000","last_became_active":"2024-05-27T08:35:07.049755+0000","last_became_peered":"2024-05-27T08:35:07.049755+0000","last_unstale":"2024-05-27T08:35:58.368861+0000","last_undegraded":"2024-05-27T08:35:58.368861+0000","last_fullsized":"2024-05-27T08:35:58.368861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:39.768128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.368191+0000","last_change":"2024-05-27T08:35:07.048281+0000","last_active":"2024-05-27T08:35:58.368191+0000","last_peered":"2024-05-27T08:35:58.368191+0000","last_clean":"2024-05-27T08:35:58.368191+0000","last_became_active":"2024-05-27T08:35:07.047879+0000","last_became_peered":"2024-05-27T08:35:07.047879+0000","last_unstale":"2024-05-27T08:35:58.368191+0000","last_undegraded":"2024-05-27T08:35:58.368191+0000","last_fullsized":"2024-05-27T08:35:58.368191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:43.631671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.370546+0000","last_change":"2024-05-27T08:35:07.050501+0000","last_active":"2024-05-27T08:35:58.370546+0000","last_peered":"2024-05-27T08:35:58.370546+0000","last_clean":"2024-05-27T08:35:58.370546+0000","last_became_active":"2024-05-27T08:35:07.050159+0000","last_became_peered":"2024-05-27T08:35:07.050159+0000","last_unstale":"2024-05-27T08:35:58.370546+0000","last_undegraded":"2024-05-27T08:35:58.370546+0000","last_fullsized":"2024-05-27T08:35:58.370546+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:17:23.293019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.181068+0000","last_change":"2024-05-27T08:35:58.372446+0000","last_active":"2024-05-27T08:36:57.181068+0000","last_peered":"2024-05-27T08:36:57.181068+0000","last_clean":"2024-05-27T08:36:57.181068+0000","last_became_active":"2024-05-27T08:35:58.372080+0000","last_became_peered":"2024-05-27T08:35:58.372080+0000","last_unstale":"2024-05-27T08:36:57.181068+0000","last_undegraded":"2024-05-27T08:36:57.181068+0000","last_fullsized":"2024-05-27T08:36:57.181068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:57.828720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.172445+0000","last_change":"2024-05-27T08:35:07.052382+0000","last_active":"2024-05-27T08:36:57.172445+0000","last_peered":"2024-05-27T08:36:57.172445+0000","last_clean":"2024-05-27T08:36:57.172445+0000","last_became_active":"2024-05-27T08:35:07.052049+0000","last_became_peered":"2024-05-27T08:35:07.052049+0000","last_unstale":"2024-05-27T08:36:57.172445+0000","last_undegraded":"2024-05-27T08:36:57.172445+0000","last_fullsized":"2024-05-27T08:36:57.172445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:08.912872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.369555+0000","last_change":"2024-05-27T08:35:07.043810+0000","last_active":"2024-05-27T08:35:58.369555+0000","last_peered":"2024-05-27T08:35:58.369555+0000","last_clean":"2024-05-27T08:35:58.369555+0000","last_became_active":"2024-05-27T08:35:07.043449+0000","last_became_peered":"2024-05-27T08:35:07.043449+0000","last_unstale":"2024-05-27T08:35:58.369555+0000","last_undegraded":"2024-05-27T08:35:58.369555+0000","last_fullsized":"2024-05-27T08:35:58.369555+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:18.089405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.176161+0000","last_change":"2024-05-27T08:35:58.371407+0000","last_active":"2024-05-27T08:36:57.176161+0000","last_peered":"2024-05-27T08:36:57.176161+0000","last_clean":"2024-05-27T08:36:57.176161+0000","last_became_active":"2024-05-27T08:35:58.370963+0000","last_became_peered":"2024-05-27T08:35:58.370963+0000","last_unstale":"2024-05-27T08:36:57.176161+0000","last_undegraded":"2024-05-27T08:36:57.176161+0000","last_fullsized":"2024-05-27T08:36:57.176161+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:23.458573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.175849+0000","last_change":"2024-05-27T08:35:07.049960+0000","last_active":"2024-05-27T08:36:57.175849+0000","last_peered":"2024-05-27T08:36:57.175849+0000","last_clean":"2024-05-27T08:36:57.175849+0000","last_became_active":"2024-05-27T08:35:07.049595+0000","last_became_peered":"2024-05-27T08:35:07.049595+0000","last_unstale":"2024-05-27T08:36:57.175849+0000","last_undegraded":"2024-05-27T08:36:57.175849+0000","last_fullsized":"2024-05-27T08:36:57.175849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:18.182658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.174624+0000","last_change":"2024-05-27T08:35:07.043790+0000","last_active":"2024-05-27T08:36:57.174624+0000","last_peered":"2024-05-27T08:36:57.174624+0000","last_clean":"2024-05-27T08:36:57.174624+0000","last_became_active":"2024-05-27T08:35:07.043476+0000","last_became_peered":"2024-05-27T08:35:07.043476+0000","last_unstale":"2024-05-27T08:36:57.174624+0000","last_undegraded":"2024-05-27T08:36:57.174624+0000","last_fullsized":"2024-05-27T08:36:57.174624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:01.626979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:35:58.374792+0000","last_change":"2024-05-27T08:35:07.044928+0000","last_active":"2024-05-27T08:35:58.374792+0000","last_peered":"2024-05-27T08:35:58.374792+0000","last_clean":"2024-05-27T08:35:58.374792+0000","last_became_active":"2024-05-27T08:35:07.044577+0000","last_became_peered":"2024-05-27T08:35:07.044577+0000","last_unstale":"2024-05-27T08:35:58.374792+0000","last_undegraded":"2024-05-27T08:35:58.374792+0000","last_fullsized":"2024-05-27T08:35:58.374792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:36.560525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.180537+0000","last_change":"2024-05-27T08:35:07.045067+0000","last_active":"2024-05-27T08:36:57.180537+0000","last_peered":"2024-05-27T08:36:57.180537+0000","last_clean":"2024-05-27T08:36:57.180537+0000","last_became_active":"2024-05-27T08:35:07.044759+0000","last_became_peered":"2024-05-27T08:35:07.044759+0000","last_unstale":"2024-05-27T08:36:57.180537+0000","last_undegraded":"2024-05-27T08:36:57.180537+0000","last_fullsized":"2024-05-27T08:36:57.180537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:08.440135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-27T08:36:57.174343+0000","last_change":"2024-05-27T08:35:07.048186+0000","last_active":"2024-05-27T08:36:57.174343+0000","last_peered":"2024-05-27T08:36:57.174343+0000","last_clean":"2024-05-27T08:36:57.174343+0000","last_became_active":"2024-05-27T08:35:07.047828+0000","last_became_peered":"2024-05-27T08:35:07.047828+0000","last_unstale":"2024-05-27T08:36:57.174343+0000","last_undegraded":"2024-05-27T08:36:57.174343+0000","last_fullsized":"2024-05-27T08:36:57.174343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:09.928234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":254,"ondisk_log_size":254,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67248,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":961,"num_read_kb":550,"num_write":249,"num_write_kb":138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":448717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1099317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":448065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135925,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":942080,"data_stored":447189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1552384,"data_stored":1041352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":447782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":509129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":449121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":508956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117070,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":447607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":448287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691909,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32704,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712448,"statfs":{"total":95995035648,"available":95961546752,"internally_reserved":0,"allocated":1564672,"data_stored":1040780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T08:37:00.037 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T08:37:00.037 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-27T08:37:00.037 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-27T08:37:00.037 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-05-27T08:37:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:00.962 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:00 smithi077 ceph-mon[32007]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-27T08:37:00.962 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:00 smithi077 ceph-mon[32007]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:37:01.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:00 smithi179 ceph-mon[36481]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-27T08:37:01.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:00 smithi179 ceph-mon[36481]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:37:01.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:00 smithi092 ceph-mon[36049]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-05-27T08:37:01.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:00 smithi092 ceph-mon[36049]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:37:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.925+0000 7f36a3a81700 1 -- 172.21.15.77:0/1758765703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103fc0 msgr2=0x7f369c1103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.925+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1758765703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103fc0 0x7f369c1103b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f368c0099c0 tx=0x7f368c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.925+0000 7f36a3a81700 1 -- 172.21.15.77:0/1758765703 shutdown_connections 2024-05-27T08:37:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.925+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1758765703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103fc0 0x7f369c1103b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.925+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1758765703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f369c1095e0 0x7f369c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.925+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1758765703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c103600 0x7f369c103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.925+0000 7f36a3a81700 1 -- 172.21.15.77:0/1758765703 >> 172.21.15.77:0/1758765703 conn(0x7f369c07a700 msgr2=0x7f369c07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:01.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 -- 172.21.15.77:0/1758765703 shutdown_connections 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 -- 172.21.15.77:0/1758765703 wait complete. 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 Processor -- start 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 -- start start 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103600 0x7f369c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f369c103fc0 0x7f369c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c1095e0 0x7f369c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f369c113740 con 0x7f369c103600 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1135c0 con 0x7f369c103fc0 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a3a81700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f369c1138c0 con 0x7f369c1095e0 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.926+0000 7f36a181d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103600 0x7f369c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a181d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103600 0x7f369c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60952/0 (socket says 172.21.15.77:60952) 2024-05-27T08:37:01.930 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a181d700 1 -- 172.21.15.77:0/1105759371 learned_addr learned my addr 172.21.15.77:0/1105759371 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a101c700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f369c103fc0 0x7f369c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a201e700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c1095e0 0x7f369c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a181d700 1 -- 172.21.15.77:0/1105759371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c1095e0 msgr2=0x7f369c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a181d700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c1095e0 0x7f369c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a181d700 1 -- 172.21.15.77:0/1105759371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f369c103fc0 msgr2=0x7f369c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a181d700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f369c103fc0 0x7f369c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a181d700 1 -- 172.21.15.77:0/1105759371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369c19a020 con 0x7f369c103600 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a101c700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f369c103fc0 0x7f369c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a201e700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c1095e0 0x7f369c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a181d700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103600 0x7f369c19f700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f369400ba50 tx=0x7f369400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f369affd700 1 -- 172.21.15.77:0/1105759371 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694019070 con 0x7f369c103600 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f369affd700 1 -- 172.21.15.77:0/1105759371 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3694014440 con 0x7f369c103600 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f369affd700 1 -- 172.21.15.77:0/1105759371 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36940075d0 con 0x7f369c103600 2024-05-27T08:37:01.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f369c19a310 con 0x7f369c103600 2024-05-27T08:37:01.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.927+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f369c1a65d0 con 0x7f369c103600 2024-05-27T08:37:01.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.928+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f369c06a230 con 0x7f369c103600 2024-05-27T08:37:01.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.930+0000 7f369affd700 1 -- 172.21.15.77:0/1105759371 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f369401c070 con 0x7f369c103600 2024-05-27T08:37:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.931+0000 7f369affd700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f36840b2900 0x7f36840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.931+0000 7f36a101c700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f36840b2900 0x7f36840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.931+0000 7f369affd700 1 -- 172.21.15.77:0/1105759371 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f36940e1ac0 con 0x7f369c103600 2024-05-27T08:37:01.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.932+0000 7f36a101c700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f36840b2900 0x7f36840b4dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3690005fd0 tx=0x7f3690005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:01.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:01.935+0000 7f369affd700 1 -- 172.21.15.77:0/1105759371 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36940a97e0 con 0x7f369c103600 2024-05-27T08:37:02.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.184+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f369c19adf0 con 0x7f369c103600 2024-05-27T08:37:02.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.184+0000 7f369affd700 1 -- 172.21.15.77:0/1105759371 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f36940a90e0 con 0x7f369c103600 2024-05-27T08:37:02.186 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:02.186 INFO:teuthology.orchestra.run.smithi077.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-27T08:37:02.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f36840b2900 msgr2=0x7f36840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:02.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f36840b2900 0x7f36840b4dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3690005fd0 tx=0x7f3690005e30 comp rx=0 tx=0).stop 2024-05-27T08:37:02.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103600 msgr2=0x7f369c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:02.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103600 0x7f369c19f700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f369400ba50 tx=0x7f369400be10 comp rx=0 tx=0).stop 2024-05-27T08:37:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 shutdown_connections 2024-05-27T08:37:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f36840b2900 0x7f36840b4dc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f369c1095e0 0x7f369c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f369c103600 0x7f369c19f700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 --2- 172.21.15.77:0/1105759371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f369c103fc0 0x7f369c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 >> 172.21.15.77:0/1105759371 conn(0x7f369c07a700 msgr2=0x7f369c101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 shutdown_connections 2024-05-27T08:37:02.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:02.187+0000 7f36a3a81700 1 -- 172.21.15.77:0/1105759371 wait complete. 2024-05-27T08:37:02.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:02 smithi077 ceph-mon[32007]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-27T08:37:02.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:02 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1105759371' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T08:37:03.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:02 smithi179 ceph-mon[36481]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-27T08:37:03.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:02 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1105759371' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T08:37:03.134 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-27T08:37:03.137 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:37:03.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:02 smithi092 ceph-mon[36049]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-27T08:37:03.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:02 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1105759371' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-27T08:37:03.381 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:04.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 -- 172.21.15.77:0/687877869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641037f0 msgr2=0x7f6764103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:04.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 --2- 172.21.15.77:0/687877869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641037f0 0x7f6764103bd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67540099c0 tx=0x7f6754009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:04.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 -- 172.21.15.77:0/687877869 shutdown_connections 2024-05-27T08:37:04.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 --2- 172.21.15.77:0/687877869 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67641088f0 0x7f6764110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:04.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 --2- 172.21.15.77:0/687877869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641037f0 0x7f6764103bd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:04.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 --2- 172.21.15.77:0/687877869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67641041a0 0x7f67641081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 -- 172.21.15.77:0/687877869 >> 172.21.15.77:0/687877869 conn(0x7f676407a6a0 msgr2=0x7f676407aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 -- 172.21.15.77:0/687877869 shutdown_connections 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.990+0000 7f6769597700 1 -- 172.21.15.77:0/687877869 wait complete. 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6769597700 1 Processor -- start 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6769597700 1 -- start start 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6769597700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67641037f0 0x7f676419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6769597700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67641041a0 0x7f676419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6769597700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641088f0 0x7f6764199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6769597700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6764113730 con 0x7f67641088f0 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6769597700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f67641135b0 con 0x7f67641037f0 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6769597700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67641138b0 con 0x7f67641041a0 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f67637fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641088f0 0x7f6764199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:04.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f67637fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641088f0 0x7f6764199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60978/0 (socket says 172.21.15.77:60978) 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f67637fe700 1 -- 172.21.15.77:0/125737852 learned_addr learned my addr 172.21.15.77:0/125737852 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.991+0000 7f6762ffd700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67641037f0 0x7f676419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f67627fc700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67641041a0 0x7f676419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f67637fe700 1 -- 172.21.15.77:0/125737852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67641041a0 msgr2=0x7f676419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f67637fe700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67641041a0 0x7f676419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f67637fe700 1 -- 172.21.15.77:0/125737852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67641037f0 msgr2=0x7f676419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f67637fe700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67641037f0 0x7f676419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f67637fe700 1 -- 172.21.15.77:0/125737852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676419a170 con 0x7f67641088f0 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f67627fc700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67641041a0 0x7f676419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f67637fe700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641088f0 0x7f6764199a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f675c007d70 tx=0x7f675c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f674bfff700 1 -- 172.21.15.77:0/125737852 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c016020 con 0x7f67641088f0 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f674bfff700 1 -- 172.21.15.77:0/125737852 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f675c00ad40 con 0x7f67641088f0 2024-05-27T08:37:04.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676419a460 con 0x7f67641088f0 2024-05-27T08:37:04.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f674bfff700 1 -- 172.21.15.77:0/125737852 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c0115e0 con 0x7f67641088f0 2024-05-27T08:37:04.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.992+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67641a66d0 con 0x7f67641088f0 2024-05-27T08:37:04.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.993+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f676410bb30 con 0x7f67641088f0 2024-05-27T08:37:04.997 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:04 smithi077 ceph-mon[32007]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T08:37:04.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.994+0000 7f674bfff700 1 -- 172.21.15.77:0/125737852 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f675c019020 con 0x7f67641088f0 2024-05-27T08:37:04.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.995+0000 7f674bfff700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f674c0b2880 0x7f674c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:04.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.995+0000 7f6762ffd700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f674c0b2880 0x7f674c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:04.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.995+0000 7f674bfff700 1 -- 172.21.15.77:0/125737852 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f675c0e2590 con 0x7f67641088f0 2024-05-27T08:37:04.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.996+0000 7f6762ffd700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f674c0b2880 0x7f674c0b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67641084a0 tx=0x7f675400b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:05.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:04.999+0000 7f674bfff700 1 -- 172.21.15.77:0/125737852 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f675c020020 con 0x7f67641088f0 2024-05-27T08:37:05.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:04 smithi179 ceph-mon[36481]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T08:37:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:04 smithi092 ceph-mon[36049]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T08:37:05.247 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.246+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f676406a230 con 0x7f67641088f0 2024-05-27T08:37:05.248 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.246+0000 7f674bfff700 1 -- 172.21.15.77:0/125737852 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11927 (secure 0 0 0) 0x7f675c0aa2b0 con 0x7f67641088f0 2024-05-27T08:37:05.248 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:05.248 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:35:26.135167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24433,24412,24410,24418,24434]},"id":1}]} 2024-05-27T08:37:05.250 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f674c0b2880 msgr2=0x7f674c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f674c0b2880 0x7f674c0b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67641084a0 tx=0x7f675400b870 comp rx=0 tx=0).stop 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641088f0 msgr2=0x7f6764199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641088f0 0x7f6764199a10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f675c007d70 tx=0x7f675c00a340 comp rx=0 tx=0).stop 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 shutdown_connections 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67641088f0 0x7f6764199a10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67641037f0 0x7f676419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f674c0b2880 0x7f674c0b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 --2- 172.21.15.77:0/125737852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67641041a0 0x7f676419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 >> 172.21.15.77:0/125737852 conn(0x7f676407a6a0 msgr2=0x7f6764102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 shutdown_connections 2024-05-27T08:37:05.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:05.249+0000 7f6769597700 1 -- 172.21.15.77:0/125737852 wait complete. 2024-05-27T08:37:05.252 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-05-27T08:37:05.807 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:37:06.048 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:06.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:05 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/125737852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:05 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/125737852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:06.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:05 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/125737852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:07.081 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:06 smithi077 ceph-mon[32007]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T08:37:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:06 smithi179 ceph-mon[36481]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T08:37:07.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:06 smithi092 ceph-mon[36049]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-05-27T08:37:07.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.684+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2919075751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0103850 msgr2=0x7fc1c0103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:07.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.684+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2919075751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0103850 0x7fc1c0103c30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0099c0 tx=0x7fc1bc009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.684+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2919075751 shutdown_connections 2024-05-27T08:37:07.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.684+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2919075751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1c0108920 0x7fc1c01104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.684+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2919075751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0103850 0x7fc1c0103c30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.684+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2919075751 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c0104200 0x7fc1c01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.684+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2919075751 >> 172.21.15.77:0/2919075751 conn(0x7fc1c007a5d0 msgr2=0x7fc1c007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:07.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.684+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2919075751 shutdown_connections 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2919075751 wait complete. 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 Processor -- start 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 -- start start 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c0103850 0x7fc1c0075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1c0104200 0x7fc1c0075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0108920 0x7fc1c00762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c0113770 con 0x7fc1c0108920 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01135f0 con 0x7fc1c0103850 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.685+0000 7fc1c72b6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01138f0 con 0x7fc1c0104200 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0108920 0x7fc1c00762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:07.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0108920 0x7fc1c00762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60984/0 (socket says 172.21.15.77:60984) 2024-05-27T08:37:07.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 -- 172.21.15.77:0/2173218567 learned_addr learned my addr 172.21.15.77:0/2173218567 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:07.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5052700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c0103850 0x7fc1c0075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:07.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c4851700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1c0104200 0x7fc1c0075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:07.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 -- 172.21.15.77:0/2173218567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1c0104200 msgr2=0x7fc1c0075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:07.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1c0104200 0x7fc1c0075d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 -- 172.21.15.77:0/2173218567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c0103850 msgr2=0x7fc1c0075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:07.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c0103850 0x7fc1c0075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 -- 172.21.15.77:0/2173218567 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c01ac550 con 0x7fc1c0108920 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5052700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c0103850 0x7fc1c0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c5853700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0108920 0x7fc1c00762c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800b970 tx=0x7fc1b800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1b67fc700 1 -- 172.21.15.77:0/2173218567 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8019070 con 0x7fc1c0108920 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1b67fc700 1 -- 172.21.15.77:0/2173218567 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b8014440 con 0x7fc1c0108920 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c01ac7e0 con 0x7fc1c0108920 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1b67fc700 1 -- 172.21.15.77:0/2173218567 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800ce60 con 0x7fc1c0108920 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.686+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c01acd20 con 0x7fc1c0108920 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.687+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c010bb60 con 0x7fc1c0108920 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.688+0000 7fc1b67fc700 1 -- 172.21.15.77:0/2173218567 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc1b801c070 con 0x7fc1c0108920 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.689+0000 7fc1b67fc700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1ac0b2880 0x7fc1ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:07.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.689+0000 7fc1c5052700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1ac0b2880 0x7fc1ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:07.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.689+0000 7fc1b67fc700 1 -- 172.21.15.77:0/2173218567 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc1b80e16a0 con 0x7fc1c0108920 2024-05-27T08:37:07.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.690+0000 7fc1c5052700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1ac0b2880 0x7fc1ac0b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc003800 tx=0x7fc1bc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:07.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.694+0000 7fc1b67fc700 1 -- 172.21.15.77:0/2173218567 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc1b80a93c0 con 0x7fc1c0108920 2024-05-27T08:37:07.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.945+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1c010b540 con 0x7fc1c0108920 2024-05-27T08:37:07.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.946+0000 7fc1b67fc700 1 -- 172.21.15.77:0/2173218567 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11927 (secure 0 0 0) 0x7fc1b80a8cc0 con 0x7fc1c0108920 2024-05-27T08:37:07.948 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:07.948 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:35:26.135167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24433,24412,24410,24418,24434]},"id":1}]} 2024-05-27T08:37:07.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1ac0b2880 msgr2=0x7fc1ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:07.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1ac0b2880 0x7fc1ac0b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc003800 tx=0x7fc1bc005fd0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0108920 msgr2=0x7fc1c00762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:07.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0108920 0x7fc1c00762c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800b970 tx=0x7fc1b800bd30 comp rx=0 tx=0).stop 2024-05-27T08:37:07.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 shutdown_connections 2024-05-27T08:37:07.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0108920 0x7fc1c00762c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1c0103850 0x7fc1c0075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1ac0b2880 0x7fc1ac0b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 --2- 172.21.15.77:0/2173218567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1c0104200 0x7fc1c0075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:07.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.948+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 >> 172.21.15.77:0/2173218567 conn(0x7fc1c007a5d0 msgr2=0x7fc1c0101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:07.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.949+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 shutdown_connections 2024-05-27T08:37:07.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:07.949+0000 7fc1c72b6700 1 -- 172.21.15.77:0/2173218567 wait complete. 2024-05-27T08:37:07.951 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-05-27T08:37:08.660 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:37:08.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:08 smithi077 ceph-mon[32007]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-27T08:37:08.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:08 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2173218567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:08.903 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:09.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:08 smithi179 ceph-mon[36481]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-27T08:37:09.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:08 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2173218567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:08 smithi092 ceph-mon[36049]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-27T08:37:09.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:08 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2173218567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:10.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 -- 172.21.15.77:0/3951713291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa20101f00 msgr2=0x7ffa20102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:10.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/3951713291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa20101f00 0x7ffa20102380 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009990 tx=0x7ffa10009ca0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 -- 172.21.15.77:0/3951713291 shutdown_connections 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/3951713291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa201028c0 0x7ffa20110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/3951713291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa20108620 0x7ffa20108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/3951713291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa20101f00 0x7ffa20102380 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 -- 172.21.15.77:0/3951713291 >> 172.21.15.77:0/3951713291 conn(0x7ffa20077780 msgr2=0x7ffa20077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 -- 172.21.15.77:0/3951713291 shutdown_connections 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.524+0000 7ffa27a1e700 1 -- 172.21.15.77:0/3951713291 wait complete. 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa27a1e700 1 Processor -- start 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa27a1e700 1 -- start start 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa27a1e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa20101f00 0x7ffa201030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa27a1e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa201028c0 0x7ffa20103620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa27a1e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20108620 0x7ffa20103b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa27a1e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20113800 con 0x7ffa201028c0 2024-05-27T08:37:10.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa27a1e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20113680 con 0x7ffa20108620 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa27a1e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20113980 con 0x7ffa20101f00 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa201028c0 0x7ffa20103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa201028c0 0x7ffa20103620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52318/0 (socket says 172.21.15.77:52318) 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 -- 172.21.15.77:0/884903016 learned_addr learned my addr 172.21.15.77:0/884903016 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa257ba700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa20101f00 0x7ffa201030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa25fbb700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20108620 0x7ffa20103b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 -- 172.21.15.77:0/884903016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa20101f00 msgr2=0x7ffa201030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa20101f00 0x7ffa201030e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 -- 172.21.15.77:0/884903016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20108620 msgr2=0x7ffa20103b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20108620 0x7ffa20103b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 -- 172.21.15.77:0/884903016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa201a8220 con 0x7ffa201028c0 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa25fbb700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20108620 0x7ffa20103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:10.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.525+0000 7ffa24fb9700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa201028c0 0x7ffa20103620 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005910 tx=0x7ffa100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.526+0000 7ffa167fc700 1 -- 172.21.15.77:0/884903016 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10027070 con 0x7ffa201028c0 2024-05-27T08:37:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.526+0000 7ffa167fc700 1 -- 172.21.15.77:0/884903016 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa10003f20 con 0x7ffa201028c0 2024-05-27T08:37:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.526+0000 7ffa167fc700 1 -- 172.21.15.77:0/884903016 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10022c70 con 0x7ffa201028c0 2024-05-27T08:37:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.526+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa201a84b0 con 0x7ffa201028c0 2024-05-27T08:37:10.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.526+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201a8900 con 0x7ffa201028c0 2024-05-27T08:37:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.527+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa2006a230 con 0x7ffa201028c0 2024-05-27T08:37:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.528+0000 7ffa167fc700 1 -- 172.21.15.77:0/884903016 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffa1002a070 con 0x7ffa201028c0 2024-05-27T08:37:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.529+0000 7ffa167fc700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffa0c0b2880 0x7ffa0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.529+0000 7ffa257ba700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffa0c0b2880 0x7ffa0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.529+0000 7ffa167fc700 1 -- 172.21.15.77:0/884903016 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ffa100e37e0 con 0x7ffa201028c0 2024-05-27T08:37:10.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.529+0000 7ffa257ba700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffa0c0b2880 0x7ffa0c0b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa20078ce0 tx=0x7ffa1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:10.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.532+0000 7ffa167fc700 1 -- 172.21.15.77:0/884903016 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa10031070 con 0x7ffa201028c0 2024-05-27T08:37:10.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.743+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa2002e560 con 0x7ffa201028c0 2024-05-27T08:37:10.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.744+0000 7ffa167fc700 1 -- 172.21.15.77:0/884903016 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22676 (secure 0 0 0) 0x7ffa10013070 con 0x7ffa201028c0 2024-05-27T08:37:10.746 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:10.746 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":85,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:35:58.361436+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:37:10.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffa0c0b2880 msgr2=0x7ffa0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:10.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffa0c0b2880 0x7ffa0c0b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa20078ce0 tx=0x7ffa1c00b410 comp rx=0 tx=0).stop 2024-05-27T08:37:10.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa201028c0 msgr2=0x7ffa20103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:10.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa201028c0 0x7ffa20103620 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005910 tx=0x7ffa100047b0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 shutdown_connections 2024-05-27T08:37:10.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffa20108620 0x7ffa20103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffa0c0b2880 0x7ffa0c0b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa20101f00 0x7ffa201030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 --2- 172.21.15.77:0/884903016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa201028c0 0x7ffa20103620 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:10.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 >> 172.21.15.77:0/884903016 conn(0x7ffa20077780 msgr2=0x7ffa20100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:10.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 shutdown_connections 2024-05-27T08:37:10.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:10.747+0000 7ffa27a1e700 1 -- 172.21.15.77:0/884903016 wait complete. 2024-05-27T08:37:11.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:10 smithi179 ceph-mon[36481]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T08:37:11.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:10 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/884903016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:37:11.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:10 smithi077 ceph-mon[32007]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T08:37:11.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:10 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/884903016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:37:11.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:10 smithi092 ceph-mon[36049]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-05-27T08:37:11.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:10 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/884903016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:37:11.453 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:37:11.697 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:13.065 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:12 smithi077 ceph-mon[32007]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-27T08:37:13.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:12 smithi179 ceph-mon[36481]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-27T08:37:13.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:12 smithi092 ceph-mon[36049]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-27T08:37:13.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 -- 172.21.15.77:0/2623336355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 msgr2=0x7fc820103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:13.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 --2- 172.21.15.77:0/2623336355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 0x7fc820103bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc81000b0d0 tx=0x7fc81000b3e0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.309 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 -- 172.21.15.77:0/2623336355 shutdown_connections 2024-05-27T08:37:13.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 --2- 172.21.15.77:0/2623336355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8201088f0 0x7fc820110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 --2- 172.21.15.77:0/2623336355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 0x7fc820103bd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 --2- 172.21.15.77:0/2623336355 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8201041a0 0x7fc8201081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 -- 172.21.15.77:0/2623336355 >> 172.21.15.77:0/2623336355 conn(0x7fc82007a6a0 msgr2=0x7fc82007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:13.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 -- 172.21.15.77:0/2623336355 shutdown_connections 2024-05-27T08:37:13.310 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 -- 172.21.15.77:0/2623336355 wait complete. 2024-05-27T08:37:13.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.308+0000 7fc825c39700 1 Processor -- start 2024-05-27T08:37:13.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc825c39700 1 -- start start 2024-05-27T08:37:13.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc825c39700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 0x7fc82019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:13.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc825c39700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8201041a0 0x7fc82019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:13.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc825c39700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8201088f0 0x7fc8201999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:13.312 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc825c39700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc820113740 con 0x7fc8201037f0 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc825c39700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201135c0 con 0x7fc8201088f0 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc825c39700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8201138c0 con 0x7fc8201041a0 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 0x7fc82019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 0x7fc82019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52332/0 (socket says 172.21.15.77:52332) 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 -- 172.21.15.77:0/2571651002 learned_addr learned my addr 172.21.15.77:0/2571651002 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81effd700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8201041a0 0x7fc82019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81ffff700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8201088f0 0x7fc8201999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 -- 172.21.15.77:0/2571651002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8201041a0 msgr2=0x7fc82019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8201041a0 0x7fc82019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 -- 172.21.15.77:0/2571651002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8201088f0 msgr2=0x7fc8201999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8201088f0 0x7fc8201999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 -- 172.21.15.77:0/2571651002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82019a120 con 0x7fc8201037f0 2024-05-27T08:37:13.313 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81effd700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8201041a0 0x7fc82019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81ffff700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8201088f0 0x7fc8201999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.309+0000 7fc81f7fe700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 0x7fc82019f7b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc810007a20 tx=0x7fc810007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.310+0000 7fc81cff9700 1 -- 172.21.15.77:0/2571651002 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81001f070 con 0x7fc8201037f0 2024-05-27T08:37:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.310+0000 7fc81cff9700 1 -- 172.21.15.77:0/2571651002 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc810003c00 con 0x7fc8201037f0 2024-05-27T08:37:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.310+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82019a3b0 con 0x7fc8201037f0 2024-05-27T08:37:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.310+0000 7fc81cff9700 1 -- 172.21.15.77:0/2571651002 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81001b730 con 0x7fc8201037f0 2024-05-27T08:37:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.310+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8201a6680 con 0x7fc8201037f0 2024-05-27T08:37:13.314 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.311+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82006a230 con 0x7fc8201037f0 2024-05-27T08:37:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.312+0000 7fc81cff9700 1 -- 172.21.15.77:0/2571651002 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc810022070 con 0x7fc8201037f0 2024-05-27T08:37:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.312+0000 7fc81cff9700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8080b2880 0x7fc8080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.313+0000 7fc81effd700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8080b2880 0x7fc8080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.313+0000 7fc81cff9700 1 -- 172.21.15.77:0/2571651002 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc81000f020 con 0x7fc8201037f0 2024-05-27T08:37:13.315 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.313+0000 7fc81effd700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8080b2880 0x7fc8080b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc814005fd0 tx=0x7fc814005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:13.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.316+0000 7fc81cff9700 1 -- 172.21.15.77:0/2571651002 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc810029020 con 0x7fc8201037f0 2024-05-27T08:37:13.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.566+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc82002e560 con 0x7fc8201037f0 2024-05-27T08:37:13.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.567+0000 7fc81cff9700 1 -- 172.21.15.77:0/2571651002 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11927 (secure 0 0 0) 0x7fc8100ab750 con 0x7fc8201037f0 2024-05-27T08:37:13.568 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:13.568 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:35:26.135167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24433,24412,24410,24418,24434]},"id":1}]} 2024-05-27T08:37:13.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8080b2880 msgr2=0x7fc8080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:13.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8080b2880 0x7fc8080b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc814005fd0 tx=0x7fc814005e20 comp rx=0 tx=0).stop 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 msgr2=0x7fc82019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 0x7fc82019f7b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc810007a20 tx=0x7fc810007b00 comp rx=0 tx=0).stop 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 shutdown_connections 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc8201088f0 0x7fc8201999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8201037f0 0x7fc82019f7b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8080b2880 0x7fc8080b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 --2- 172.21.15.77:0/2571651002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8201041a0 0x7fc82019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 >> 172.21.15.77:0/2571651002 conn(0x7fc82007a6a0 msgr2=0x7fc820101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 shutdown_connections 2024-05-27T08:37:13.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:13.569+0000 7fc825c39700 1 -- 172.21.15.77:0/2571651002 wait complete. 2024-05-27T08:37:13.572 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-05-27T08:37:14.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:13 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2571651002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:14.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:13 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2571651002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:14.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T08:35:08.767215+0000', 'modified': '2024-05-27T08:35:26.135167+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24434, 'mds_1': 24433, 'mds_2': 24418, 'mds_3': 24412, 'mds_4': 24410}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/3869213740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 3869213740}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 3869213740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6841/906525687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 906525687}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 906525687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24365': {'gid': 24365, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6833/1631050328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 1631050328}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 1631050328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6833/1008462834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 1008462834}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 1008462834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/4019780450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 4019780450}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 4019780450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/2920087591', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 2920087591}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2920087591}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.179:6835/2178851321', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 2178851321}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 2178851321}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.179:6837/2857088493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 2857088493}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 2857088493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.179:6839/2672999131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2672999131}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2672999131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/709031605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 709031605}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 709031605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24434, 'qdb_cluster': [24433, 24412, 24410, 24418, 24434]} 2024-05-27T08:37:14.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-27T08:37:14.189 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T08:37:14.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:13 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2571651002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:14.429 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:15.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:14 smithi179 ceph-mon[36481]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-27T08:37:15.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:14 smithi077 ceph-mon[32007]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-27T08:37:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:14 smithi092 ceph-mon[36049]: pgmap v353: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-05-27T08:37:15.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 -- 172.21.15.77:0/1957571235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c1084b0 msgr2=0x7f233c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 --2- 172.21.15.77:0/1957571235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c1084b0 0x7f233c108890 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2330009990 tx=0x7f2330009ca0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 -- 172.21.15.77:0/1957571235 shutdown_connections 2024-05-27T08:37:15.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 --2- 172.21.15.77:0/1957571235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c102e30 0x7f233c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 --2- 172.21.15.77:0/1957571235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c1084b0 0x7f233c108890 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 --2- 172.21.15.77:0/1957571235 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f233c102470 0x7f233c1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 -- 172.21.15.77:0/1957571235 >> 172.21.15.77:0/1957571235 conn(0x7f233c079410 msgr2=0x7f233c079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:15.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 -- 172.21.15.77:0/1957571235 shutdown_connections 2024-05-27T08:37:15.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.947+0000 7f2342297700 1 -- 172.21.15.77:0/1957571235 wait complete. 2024-05-27T08:37:15.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f2342297700 1 Processor -- start 2024-05-27T08:37:15.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f2342297700 1 -- start start 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f2342297700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c102470 0x7f233c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f2342297700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c102e30 0x7f233c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f2342297700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f233c1084b0 0x7f233c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f2342297700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f233c112640 con 0x7f233c102e30 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f2342297700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f233c1124c0 con 0x7f233c1084b0 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f2342297700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f233c1127c0 con 0x7f233c102470 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c102e30 0x7f233c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c102e30 0x7f233c19fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52356/0 (socket says 172.21.15.77:52356) 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.948+0000 7f233b7fe700 1 -- 172.21.15.77:0/1247271254 learned_addr learned my addr 172.21.15.77:0/1247271254 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:15.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f2340834700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f233c1084b0 0x7f233c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f233bfff700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c102470 0x7f233c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f233b7fe700 1 -- 172.21.15.77:0/1247271254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c102470 msgr2=0x7f233c19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c102470 0x7f233c19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f233b7fe700 1 -- 172.21.15.77:0/1247271254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f233c1084b0 msgr2=0x7f233c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f233c1084b0 0x7f233c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f233b7fe700 1 -- 172.21.15.77:0/1247271254 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c19a210 con 0x7f233c102e30 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f233bfff700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c102470 0x7f233c19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c102e30 0x7f233c19fe30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f232c00d950 tx=0x7f232c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f23397fa700 1 -- 172.21.15.77:0/1247271254 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c016070 con 0x7f233c102e30 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f23397fa700 1 -- 172.21.15.77:0/1247271254 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f232c009df0 con 0x7f233c102e30 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f2342297700 1 -- 172.21.15.77:0/1247271254 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233c19a500 con 0x7f233c102e30 2024-05-27T08:37:15.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f23397fa700 1 -- 172.21.15.77:0/1247271254 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c00f3c0 con 0x7f233c102e30 2024-05-27T08:37:15.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.949+0000 7f2342297700 1 -- 172.21.15.77:0/1247271254 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233c103fb0 con 0x7f233c102e30 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.952+0000 7f23397fa700 1 -- 172.21.15.77:0/1247271254 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f232c019070 con 0x7f233c102e30 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.952+0000 7f23397fa700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23240b2880 0x7f23240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.952+0000 7f23397fa700 1 -- 172.21.15.77:0/1247271254 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f232c0e1e30 con 0x7f233c102e30 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f233bfff700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23240b2880 0x7f23240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f234cb2db80 1 Processor -- start 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f234cb2db80 1 -- start start 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f233bfff700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23240b2880 0x7f23240b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2330000c00 tx=0x7f2330005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87defec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87dea060 0x55cf87dea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 0x55cf87df6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f234cb2db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf87d3fd20 con 0x55cf87defae0 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f234cb2db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cf87d2f8c0 con 0x55cf87dea060 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f234cb2db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cf87d33900 con 0x55cf87deaa20 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f233bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87defec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f233bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87defec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52364/0 (socket says 172.21.15.77:52364) 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.953+0000 7f233bfff700 1 -- 172.21.15.77:0/4068020266 learned_addr learned my addr 172.21.15.77:0/4068020266 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f233b7fe700 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87dea060 0x55cf87dea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f2340834700 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 0x55cf87df6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f233bfff700 1 -- 172.21.15.77:0/4068020266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 msgr2=0x55cf87df6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f233bfff700 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 0x55cf87df6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f233bfff700 1 -- 172.21.15.77:0/4068020266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87dea060 msgr2=0x55cf87dea4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f233bfff700 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87dea060 0x55cf87dea4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f233bfff700 1 -- 172.21.15.77:0/4068020266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf87cd8f60 con 0x55cf87defae0 2024-05-27T08:37:15.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f233b7fe700 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87dea060 0x55cf87dea4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:37:15.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f233bfff700 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87defec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2330003b20 tx=0x7f233001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ccb72f78ee011c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:15.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f23227fc700 1 -- 172.21.15.77:0/4068020266 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330003e30 con 0x55cf87defae0 2024-05-27T08:37:15.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f23227fc700 1 -- 172.21.15.77:0/4068020266 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2330009590 con 0x55cf87defae0 2024-05-27T08:37:15.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.954+0000 7f23227fc700 1 -- 172.21.15.77:0/4068020266 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233000f860 con 0x55cf87defae0 2024-05-27T08:37:15.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 -- 172.21.15.77:0/4068020266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 msgr2=0x55cf87defec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87defec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2330003b20 tx=0x7f233001e000 comp rx=0 tx=0).stop 2024-05-27T08:37:15.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 -- 172.21.15.77:0/4068020266 shutdown_connections 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 0x55cf87df6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87defec0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 --2- 172.21.15.77:0/4068020266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87dea060 0x55cf87dea4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 -- 172.21.15.77:0/4068020266 >> 172.21.15.77:0/4068020266 conn(0x55cf87d0d1f0 msgr2=0x55cf87ddc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 -- 172.21.15.77:0/4068020266 shutdown_connections 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 -- 172.21.15.77:0/4068020266 wait complete. 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.955+0000 7f234cb2db80 1 -- start start 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f234cb2db80 1 Processor -- start 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87df6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87deaa20 0x55cf87df3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 0x55cf87df3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f234cb2db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf87c35f50 con 0x55cf87deaa20 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f234cb2db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cf87c97d50 con 0x55cf87defae0 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f234cb2db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cf87df6e40 con 0x55cf87dea060 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f234cb2db80 10 client.0 _ll_register_callbacks cb 0x55cf87cca900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87deaa20 0x55cf87df3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87deaa20 0x55cf87df3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52380/0 (socket says 172.21.15.77:52380) 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233b7fe700 1 -- 172.21.15.77:0/1497160359 learned_addr learned my addr 172.21.15.77:0/1497160359 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f2340834700 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 0x55cf87df3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233bfff700 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87df6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233b7fe700 1 -- 172.21.15.77:0/1497160359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 msgr2=0x55cf87df6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87df6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233b7fe700 1 -- 172.21.15.77:0/1497160359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 msgr2=0x55cf87df3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 0x55cf87df3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233b7fe700 1 -- 172.21.15.77:0/1497160359 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf87c7f880 con 0x55cf87deaa20 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.956+0000 7f233bfff700 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87df6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87deaa20 0x55cf87df3790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f232c009520 tx=0x7f232c0a9cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4530eb7008da23d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f231b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f231b7fe700 1 -- 172.21.15.77:0/1497160359 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c020070 con 0x55cf87deaa20 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f231b7fe700 1 -- 172.21.15.77:0/1497160359 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f232c010710 con 0x55cf87deaa20 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f234cb2db80 1 -- 172.21.15.77:0/1497160359 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2330009590 con 0x55cf87deaa20 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f231b7fe700 1 -- 172.21.15.77:0/1497160359 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f232c0b22b0 con 0x55cf87deaa20 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f234cb2db80 10 client.14946 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f234cb2db80 1 -- 172.21.15.77:0/1497160359 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf87ce9710 con 0x55cf87deaa20 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f231b7fe700 1 -- 172.21.15.77:0/1497160359 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2330009590 con 0x55cf87deaa20 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f231b7fe700 1 -- 172.21.15.77:0/1497160359 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f232c0a3b70 con 0x55cf87deaa20 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.957+0000 7f234cb2db80 10 client.14946 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:15.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.958+0000 7f234cb2db80 10 client.14946 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.958+0000 7f234cb2db80 20 client.14946 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.958+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55cf87e038b0 0x55cf87e04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.958+0000 7f234cb2db80 4 client.14946 mds_command: new command op to 24410 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.958+0000 7f234cb2db80 1 -- 172.21.15.77:0/1497160359 --> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf87d41c70 con 0x55cf87e038b0 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.958+0000 7f233bfff700 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55cf87e038b0 0x55cf87e04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.958+0000 7f233bfff700 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55cf87e038b0 0x55cf87e04d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eb0752792f558531 server_cookie=47f472aae357f6d6 in_seq=0 out_seq=0 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.958+0000 7f231b7fe700 10 client.14946 ms_handle_connect on v2:172.21.15.92:6836/2920087591 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f231b7fe700 1 -- 172.21.15.77:0/1497160359 <== mds.4 v2:172.21.15.92:6836/2920087591 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cf87d41c70 con 0x55cf87e038b0 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f231b7fe700 10 client.14946 handle_command_reply: tid=0 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 client.14946 shutdown 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 -- 172.21.15.77:0/1497160359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87deaa20 msgr2=0x55cf87df3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87deaa20 0x55cf87df3790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f232c009520 tx=0x7f232c0a9cb0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 -- 172.21.15.77:0/1497160359 shutdown_connections 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 0x55cf87df3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87df6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55cf87e038b0 0x55cf87e04d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1497160359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87deaa20 0x55cf87df3790 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 -- 172.21.15.77:0/1497160359 >> 172.21.15.77:0/1497160359 conn(0x55cf87d0d1f0 msgr2=0x55cf87ddbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f231b7fe700 0 client.14946 ms_handle_reset on v2:172.21.15.92:6836/2920087591 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 -- 172.21.15.77:0/1497160359 shutdown_connections 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.959+0000 7f234cb2db80 1 -- 172.21.15.77:0/1497160359 wait complete. 2024-05-27T08:37:15.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.960+0000 7f234cb2db80 20 client.14946 trim_cache size 0 max 16384 2024-05-27T08:37:15.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f234cb2db80 1 Processor -- start 2024-05-27T08:37:15.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f234cb2db80 1 -- start start 2024-05-27T08:37:15.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87dea060 0x55cf87ddc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 0x55cf87dee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 0x55cf87deedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f234cb2db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e03060 con 0x55cf87dea060 2024-05-27T08:37:15.983 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f234cb2db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e03730 con 0x55cf87defae0 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f234cb2db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e03e00 con 0x55cf87deaa20 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87dea060 0x55cf87ddc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87dea060 0x55cf87ddc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52392/0 (socket says 172.21.15.77:52392) 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 -- 172.21.15.77:0/1946038274 learned_addr learned my addr 172.21.15.77:0/1946038274 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1946038274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 0x55cf87dee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 -- 172.21.15.77:0/1946038274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 msgr2=0x55cf87dee8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 --2- 172.21.15.77:0/1946038274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 0x55cf87dee8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 -- 172.21.15.77:0/1946038274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 msgr2=0x55cf87deedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 --2- 172.21.15.77:0/1946038274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 0x55cf87deedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 -- 172.21.15.77:0/1946038274 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf87df60d0 con 0x55cf87dea060 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.981+0000 7f233bfff700 1 --2- 172.21.15.77:0/1946038274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87dea060 0x55cf87ddc220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2330009990 tx=0x7f2330026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b5c20656d3bc2c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f2321ffb700 1 -- 172.21.15.77:0/1946038274 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233003b070 con 0x55cf87dea060 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f2321ffb700 1 -- 172.21.15.77:0/1946038274 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23300054e0 con 0x55cf87dea060 2024-05-27T08:37:15.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f2321ffb700 1 -- 172.21.15.77:0/1946038274 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330036810 con 0x55cf87dea060 2024-05-27T08:37:15.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 -- 172.21.15.77:0/1946038274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87dea060 msgr2=0x55cf87ddc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1946038274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87dea060 0x55cf87ddc220 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2330009990 tx=0x7f2330026040 comp rx=0 tx=0).stop 2024-05-27T08:37:15.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 -- 172.21.15.77:0/1946038274 shutdown_connections 2024-05-27T08:37:15.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1946038274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87defae0 0x55cf87deedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1946038274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87dea060 0x55cf87ddc220 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1946038274 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87deaa20 0x55cf87dee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 -- 172.21.15.77:0/1946038274 >> 172.21.15.77:0/1946038274 conn(0x55cf87d0d1f0 msgr2=0x55cf87dddda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 -- 172.21.15.77:0/1946038274 shutdown_connections 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.982+0000 7f234cb2db80 1 -- 172.21.15.77:0/1946038274 wait complete. 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 1 -- start start 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 1 Processor -- start 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87e0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87deaa20 0x55cf87e0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87ddb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e04ba0 con 0x55cf87defae0 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e044d0 con 0x55cf87deaa20 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cf87e05270 con 0x55cf87dea060 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f234cb2db80 10 client.0 _ll_register_callbacks cb 0x55cf87cca900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:15.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f2340834700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87ddb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f2340834700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87ddb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52394/0 (socket says 172.21.15.77:52394) 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f2340834700 1 -- 172.21.15.77:0/1493696266 learned_addr learned my addr 172.21.15.77:0/1493696266 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f233bfff700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87e0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.983+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87deaa20 0x55cf87e0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f2340834700 1 -- 172.21.15.77:0/1493696266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 msgr2=0x55cf87e0e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f2340834700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87e0e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f2340834700 1 -- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87deaa20 msgr2=0x55cf87e0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f2340834700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87deaa20 0x55cf87e0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f2340834700 1 -- 172.21.15.77:0/1493696266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf87ddbc40 con 0x55cf87defae0 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f233bfff700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87e0e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f233b7fe700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87deaa20 0x55cf87e0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f2340834700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87ddb430 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f233400aee0 tx=0x7f2334009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b25795b05f6f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:15.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f231b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f231b7fe700 1 -- 172.21.15.77:0/1493696266 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233401c020 con 0x55cf87defae0 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f234cb2db80 1 -- 172.21.15.77:0/1493696266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23300054e0 con 0x55cf87defae0 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f231b7fe700 1 -- 172.21.15.77:0/1493696266 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f23340079f0 con 0x55cf87defae0 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f231b7fe700 1 -- 172.21.15.77:0/1493696266 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2334007bf0 con 0x55cf87defae0 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f234cb2db80 10 client.14952 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f234cb2db80 1 -- 172.21.15.77:0/1493696266 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cf87e0cf10 con 0x55cf87defae0 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f231b7fe700 1 -- 172.21.15.77:0/1493696266 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23300054e0 con 0x55cf87defae0 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f231b7fe700 1 -- 172.21.15.77:0/1493696266 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2334023430 con 0x55cf87defae0 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.984+0000 7f234cb2db80 10 client.14952 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.985+0000 7f234cb2db80 10 client.14952 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.985+0000 7f234cb2db80 20 client.14952 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.985+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55cf87e73b10 0x55cf87e72820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.985+0000 7f234cb2db80 4 client.14952 mds_command: new command op to 24410 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:37:15.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.985+0000 7f234cb2db80 1 -- 172.21.15.77:0/1493696266 --> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] -- command(tid 0: {"prefix": "status"}) -- 0x55cf87d3b450 con 0x55cf87e73b10 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.985+0000 7f233bfff700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55cf87e73b10 0x55cf87e72820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.985+0000 7f231b7fe700 10 client.14952 ms_handle_connect on v2:172.21.15.92:6836/2920087591 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.985+0000 7f233bfff700 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55cf87e73b10 0x55cf87e72820 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=680f77dcfed06fde server_cookie=4b4b074b286cd082 in_seq=0 out_seq=0 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f231b7fe700 1 -- 172.21.15.77:0/1493696266 <== mds.4 v2:172.21.15.92:6836/2920087591 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cf87d3b450 con 0x55cf87e73b10 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f231b7fe700 10 client.14952 handle_command_reply: tid=0 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 client.14952 shutdown 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 -- 172.21.15.77:0/1493696266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 msgr2=0x55cf87ddb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87ddb430 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f233400aee0 tx=0x7f2334009e30 comp rx=0 tx=0).stop 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 -- 172.21.15.77:0/1493696266 shutdown_connections 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55cf87e73b10 0x55cf87e72820 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf87defae0 0x55cf87ddb430 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cf87dea060 0x55cf87e0e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 --2- 172.21.15.77:0/1493696266 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55cf87deaa20 0x55cf87e0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f231b7fe700 0 client.14952 ms_handle_reset on v2:172.21.15.92:6836/2920087591 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 -- 172.21.15.77:0/1493696266 >> 172.21.15.77:0/1493696266 conn(0x55cf87d0d1f0 msgr2=0x55cf87ded4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 -- 172.21.15.77:0/1493696266 shutdown_connections 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 1 -- 172.21.15.77:0/1493696266 wait complete. 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.986+0000 7f234cb2db80 20 client.14952 trim_cache size 0 max 16384 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24410, 2024-05-27T08:37:15.995 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:37:15.996 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:37:15.996 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:37:15.996 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 112.72226278799999, 2024-05-27T08:37:15.996 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:37:15.996 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:37:15.996 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:37:15.996 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 155.21294678500001 2024-05-27T08:37:15.996 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 -- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23240b2880 msgr2=0x7f23240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23240b2880 0x7f23240b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2330000c00 tx=0x7f2330005fb0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 -- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c102e30 msgr2=0x7f233c19fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c102e30 0x7f233c19fe30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f232c00d950 tx=0x7f232c00dc60 comp rx=0 tx=0).stop 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 -- 172.21.15.77:0/1247271254 shutdown_connections 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f233c1084b0 0x7f233c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f23240b2880 0x7f23240b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f233c102470 0x7f233c19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 --2- 172.21.15.77:0/1247271254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f233c102e30 0x7f233c19fe30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 -- 172.21.15.77:0/1247271254 >> 172.21.15.77:0/1247271254 conn(0x7f233c079410 msgr2=0x7f233c06bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 -- 172.21.15.77:0/1247271254 shutdown_connections 2024-05-27T08:37:15.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:15.987+0000 7f2342297700 1 -- 172.21.15.77:0/1247271254 wait complete. 2024-05-27T08:37:16.622 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-05-27T08:37:16.870 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:17.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:16 smithi179 ceph-mon[36481]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:17.114 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:16 smithi077 ceph-mon[32007]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:17.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:16 smithi092 ceph-mon[36049]: pgmap v354: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:18.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.483+0000 7f272620c700 1 -- 172.21.15.77:0/1471316002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201037f0 msgr2=0x7f2720103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 --2- 172.21.15.77:0/1471316002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201037f0 0x7f2720103bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f27100099c0 tx=0x7f2710009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 -- 172.21.15.77:0/1471316002 shutdown_connections 2024-05-27T08:37:18.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 --2- 172.21.15.77:0/1471316002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27201088f0 0x7f2720110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 --2- 172.21.15.77:0/1471316002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201037f0 0x7f2720103bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 --2- 172.21.15.77:0/1471316002 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27201041a0 0x7f27201081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 -- 172.21.15.77:0/1471316002 >> 172.21.15.77:0/1471316002 conn(0x7f272007a6a0 msgr2=0x7f272007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 -- 172.21.15.77:0/1471316002 shutdown_connections 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 -- 172.21.15.77:0/1471316002 wait complete. 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 Processor -- start 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.484+0000 7f272620c700 1 -- start start 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f272620c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27201037f0 0x7f272019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f272620c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201041a0 0x7f272019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f272620c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27201088f0 0x7f27201999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f272620c700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2720113740 con 0x7f27201041a0 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f272620c700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f27201135c0 con 0x7f27201037f0 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f272620c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f27201138c0 con 0x7f27201088f0 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201041a0 0x7f272019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201041a0 0x7f272019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52404/0 (socket says 172.21.15.77:52404) 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 -- 172.21.15.77:0/3045529687 learned_addr learned my addr 172.21.15.77:0/3045529687 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271ffff700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27201088f0 0x7f27201999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271f7fe700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27201037f0 0x7f272019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 -- 172.21.15.77:0/3045529687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27201088f0 msgr2=0x7f27201999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27201088f0 0x7f27201999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 -- 172.21.15.77:0/3045529687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27201037f0 msgr2=0x7f272019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27201037f0 0x7f272019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 -- 172.21.15.77:0/3045529687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272019a120 con 0x7f27201041a0 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271f7fe700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27201037f0 0x7f272019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271effd700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201041a0 0x7f272019fcf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f271400c9b0 tx=0x7f271400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271cff9700 1 -- 172.21.15.77:0/3045529687 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714007b00 con 0x7f27201041a0 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f272620c700 1 -- 172.21.15.77:0/3045529687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272019a410 con 0x7f27201041a0 2024-05-27T08:37:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271cff9700 1 -- 172.21.15.77:0/3045529687 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2714007ca0 con 0x7f27201041a0 2024-05-27T08:37:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f271cff9700 1 -- 172.21.15.77:0/3045529687 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714018660 con 0x7f27201041a0 2024-05-27T08:37:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.485+0000 7f272620c700 1 -- 172.21.15.77:0/3045529687 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201a6680 con 0x7f27201041a0 2024-05-27T08:37:18.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.488+0000 7f271cff9700 1 -- 172.21.15.77:0/3045529687 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f271400f450 con 0x7f27201041a0 2024-05-27T08:37:18.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.489+0000 7f271cff9700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27080b2880 0x7f27080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.489+0000 7f271f7fe700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27080b2880 0x7f27080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.489+0000 7f271cff9700 1 -- 172.21.15.77:0/3045529687 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f27140e21f0 con 0x7f27201041a0 2024-05-27T08:37:18.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27080b2880 0x7f27080b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f271000b580 tx=0x7f2710009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:18.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f2730aa2b80 1 Processor -- start 2024-05-27T08:37:18.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f2730aa2b80 1 -- start start 2024-05-27T08:37:18.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92f9810 0x55d8c92f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 0x55d8c92fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fdf10 0x55d8c92fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9258c40 con 0x55d8c92f9810 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d8c92487e0 con 0x55d8c92fa1c0 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8c924c820 con 0x55d8c92fdf10 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92f9810 0x55d8c92f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92f9810 0x55d8c92f9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52420/0 (socket says 172.21.15.77:52420) 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 -- 172.21.15.77:0/1115283461 learned_addr learned my addr 172.21.15.77:0/1115283461 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271effd700 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 0x55d8c92fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271ffff700 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fdf10 0x55d8c92fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 -- 172.21.15.77:0/1115283461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fdf10 msgr2=0x55d8c92fe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:18.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fdf10 0x55d8c92fe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 -- 172.21.15.77:0/1115283461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 msgr2=0x55d8c92fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 0x55d8c92fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.490+0000 7f271f7fe700 1 -- 172.21.15.77:0/1115283461 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c9198880 con 0x55d8c92f9810 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.491+0000 7f271f7fe700 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92f9810 0x55d8c92f9bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2710003800 tx=0x7f271001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a49ebeacd188415 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.491+0000 7f2705ffb700 1 -- 172.21.15.77:0/1115283461 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710005530 con 0x55d8c92f9810 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.491+0000 7f2705ffb700 1 -- 172.21.15.77:0/1115283461 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27100056d0 con 0x55d8c92f9810 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.491+0000 7f2705ffb700 1 -- 172.21.15.77:0/1115283461 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271002ea10 con 0x55d8c92f9810 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1115283461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92f9810 msgr2=0x55d8c92f9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92f9810 0x55d8c92f9bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2710003800 tx=0x7f271001e000 comp rx=0 tx=0).stop 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1115283461 shutdown_connections 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fdf10 0x55d8c92fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92f9810 0x55d8c92f9bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1115283461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 0x55d8c92fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1115283461 >> 172.21.15.77:0/1115283461 conn(0x55d8c9227030 msgr2=0x55d8c92f3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1115283461 shutdown_connections 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1115283461 wait complete. 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 -- start start 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.492+0000 7f2730aa2b80 1 Processor -- start 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92f9810 0x55d8c9305490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fa1c0 0x55d8c9302800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c9302d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8c914ef50 con 0x55d8c92fdf10 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d8c91ea770 con 0x55d8c92f9810 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9306070 con 0x55d8c92fa1c0 2024-05-27T08:37:18.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f2730aa2b80 10 client.0 _ll_register_callbacks cb 0x55d8c91e38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c9302d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c9302d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52430/0 (socket says 172.21.15.77:52430) 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 -- 172.21.15.77:0/3550966514 learned_addr learned my addr 172.21.15.77:0/3550966514 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271f7fe700 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92f9810 0x55d8c9305490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271effd700 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fa1c0 0x55d8c9302800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 -- 172.21.15.77:0/3550966514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fa1c0 msgr2=0x55d8c9302800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fa1c0 0x55d8c9302800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 -- 172.21.15.77:0/3550966514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92f9810 msgr2=0x55d8c9305490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92f9810 0x55d8c9305490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 -- 172.21.15.77:0/3550966514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c91e7640 con 0x55d8c92fdf10 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f271ffff700 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c9302d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f271800ae70 tx=0x7f271800af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34ede6d70a1bd64 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:18.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f26feffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f26feffd700 1 -- 172.21.15.77:0/3550966514 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718007bb0 con 0x55d8c92fdf10 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f26feffd700 1 -- 172.21.15.77:0/3550966514 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2718007d50 con 0x55d8c92fdf10 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.493+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3550966514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27100056d0 con 0x55d8c92fdf10 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f26feffd700 1 -- 172.21.15.77:0/3550966514 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2718021440 con 0x55d8c92fdf10 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f2730aa2b80 10 client.14964 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3550966514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8c9314d90 con 0x55d8c92fdf10 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f26feffd700 1 -- 172.21.15.77:0/3550966514 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27100056d0 con 0x55d8c92fdf10 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f26feffd700 1 -- 172.21.15.77:0/3550966514 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f271801f400 con 0x55d8c92fdf10 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f2730aa2b80 10 client.14964 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f2730aa2b80 10 client.14964 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-27T08:37:18.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f2730aa2b80 20 client.14964 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:18.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x55d8c9317410 0x55d8c9318880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f2730aa2b80 4 client.14964 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:37:18.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.494+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3550966514 --> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8c92c5990 con 0x55d8c9317410 2024-05-27T08:37:18.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.495+0000 7f271f7fe700 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x55d8c9317410 0x55d8c9318880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.495+0000 7f271f7fe700 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x55d8c9317410 0x55d8c9318880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f3087db518cbc0bf server_cookie=2cedcca2ac774ec in_seq=0 out_seq=0 2024-05-27T08:37:18.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.495+0000 7f26feffd700 10 client.14964 ms_handle_connect on v2:172.21.15.179:6834/2178851321 2024-05-27T08:37:18.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f26feffd700 1 -- 172.21.15.77:0/3550966514 <== mds.3 v2:172.21.15.179:6834/2178851321 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d8c92c5990 con 0x55d8c9317410 2024-05-27T08:37:18.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f26feffd700 10 client.14964 handle_command_reply: tid=0 2024-05-27T08:37:18.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 client.14964 shutdown 2024-05-27T08:37:18.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3550966514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 msgr2=0x55d8c9302d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c9302d40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f271800ae70 tx=0x7f271800af50 comp rx=0 tx=0).stop 2024-05-27T08:37:18.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3550966514 shutdown_connections 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c9302d40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x55d8c9317410 0x55d8c9318880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92f9810 0x55d8c9305490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3550966514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92fa1c0 0x55d8c9302800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3550966514 >> 172.21.15.77:0/3550966514 conn(0x55d8c9227030 msgr2=0x55d8c92f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f26feffd700 0 client.14964 ms_handle_reset on v2:172.21.15.179:6834/2178851321 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3550966514 shutdown_connections 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3550966514 wait complete. 2024-05-27T08:37:18.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.496+0000 7f2730aa2b80 20 client.14964 trim_cache size 0 max 16384 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.517+0000 7f2730aa2b80 1 Processor -- start 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.517+0000 7f2730aa2b80 1 -- start start 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.517+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 0x55d8c9304740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.517+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 0x55d8c92f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.517+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c92f4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9316bc0 con 0x55d8c92fdf10 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9317290 con 0x55d8c92fa1c0 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9317960 con 0x55d8c92f9810 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c92f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c92f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52446/0 (socket says 172.21.15.77:52446) 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 -- 172.21.15.77:0/3317353330 learned_addr learned my addr 172.21.15.77:0/3317353330 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:18.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 -- 172.21.15.77:0/3317353330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 msgr2=0x55d8c9304740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 --2- 172.21.15.77:0/3317353330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 0x55d8c9304740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 -- 172.21.15.77:0/3317353330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 msgr2=0x55d8c92f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 --2- 172.21.15.77:0/3317353330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 0x55d8c92f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 -- 172.21.15.77:0/3317353330 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8c92ec560 con 0x55d8c92fdf10 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f271ffff700 1 --2- 172.21.15.77:0/3317353330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c92f4180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f271800ab90 tx=0x7f2718009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fedbf70e3db5f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f27057fa700 1 -- 172.21.15.77:0/3317353330 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271802f070 con 0x55d8c92fdf10 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f27057fa700 1 -- 172.21.15.77:0/3317353330 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f271801b070 con 0x55d8c92fdf10 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.518+0000 7f27057fa700 1 -- 172.21.15.77:0/3317353330 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27180218d0 con 0x55d8c92fdf10 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3317353330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 msgr2=0x55d8c92f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3317353330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c92f4180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f271800ab90 tx=0x7f2718009970 comp rx=0 tx=0).stop 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3317353330 shutdown_connections 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3317353330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fdf10 0x55d8c92f4180 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3317353330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 0x55d8c9304740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/3317353330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fa1c0 0x55d8c92f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3317353330 >> 172.21.15.77:0/3317353330 conn(0x55d8c9227030 msgr2=0x55d8c9304030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:18.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3317353330 shutdown_connections 2024-05-27T08:37:18.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 -- 172.21.15.77:0/3317353330 wait complete. 2024-05-27T08:37:18.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.519+0000 7f2730aa2b80 1 -- start start 2024-05-27T08:37:18.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 1 Processor -- start 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 0x55d8c931e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fa1c0 0x55d8c931e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fdf10 0x55d8c937bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9318700 con 0x55d8c92fa1c0 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9318030 con 0x55d8c92fdf10 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8c9318dd0 con 0x55d8c92f9810 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 10 client.0 _ll_register_callbacks cb 0x55d8c91e38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fa1c0 0x55d8c931e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fa1c0 0x55d8c931e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52462/0 (socket says 172.21.15.77:52462) 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 -- 172.21.15.77:0/1892664559 learned_addr learned my addr 172.21.15.77:0/1892664559 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:18.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271f7fe700 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 0x55d8c931e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271ffff700 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fdf10 0x55d8c937bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 -- 172.21.15.77:0/1892664559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 msgr2=0x55d8c931e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 0x55d8c931e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 -- 172.21.15.77:0/1892664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fdf10 msgr2=0x55d8c937bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fdf10 0x55d8c937bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 -- 172.21.15.77:0/1892664559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8c937c2b0 con 0x55d8c92fa1c0 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271ffff700 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fdf10 0x55d8c937bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f271effd700 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fa1c0 0x55d8c931e820 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f27140dfcf0 tx=0x7f27140a8f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274e9e3e97b4652f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f26feffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f26feffd700 1 -- 172.21.15.77:0/1892664559 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27140c3060 con 0x55d8c92fa1c0 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f26feffd700 1 -- 172.21.15.77:0/1892664559 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f27140a9c80 con 0x55d8c92fa1c0 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.520+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1892664559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f271801b070 con 0x55d8c92fa1c0 2024-05-27T08:37:18.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f26feffd700 1 -- 172.21.15.77:0/1892664559 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27140a9f10 con 0x55d8c92fa1c0 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f2730aa2b80 10 client.14970 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1892664559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8c93836a0 con 0x55d8c92fa1c0 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f26feffd700 1 -- 172.21.15.77:0/1892664559 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f271801b070 con 0x55d8c92fa1c0 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f26feffd700 1 -- 172.21.15.77:0/1892664559 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f27140a8a70 con 0x55d8c92fa1c0 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f2730aa2b80 10 client.14970 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f2730aa2b80 10 client.14970 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f2730aa2b80 20 client.14970 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x55d8c9388270 0x55d8c9386f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f2730aa2b80 4 client.14970 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1892664559 --> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] -- command(tid 0: {"prefix": "status"}) -- 0x55d8c925ab90 con 0x55d8c9388270 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.521+0000 7f271f7fe700 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x55d8c9388270 0x55d8c9386f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f271f7fe700 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x55d8c9388270 0x55d8c9386f80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6748e9fe2ad24f9c server_cookie=9b225cd2ed7138aa in_seq=0 out_seq=0 2024-05-27T08:37:18.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f26feffd700 10 client.14970 ms_handle_connect on v2:172.21.15.179:6834/2178851321 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f26feffd700 1 -- 172.21.15.77:0/1892664559 <== mds.3 v2:172.21.15.179:6834/2178851321 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d8c925ab90 con 0x55d8c9388270 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f26feffd700 10 client.14970 handle_command_reply: tid=0 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 client.14970 shutdown 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1892664559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fa1c0 msgr2=0x55d8c931e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fa1c0 0x55d8c931e820 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f27140dfcf0 tx=0x7f27140a8f80 comp rx=0 tx=0).stop 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1892664559 shutdown_connections 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x55d8c9388270 0x55d8c9386f80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55d8c92fdf10 0x55d8c937bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8c92f9810 0x55d8c931e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 --2- 172.21.15.77:0/1892664559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d8c92fa1c0 0x55d8c931e820 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1892664559 >> 172.21.15.77:0/1892664559 conn(0x55d8c9227030 msgr2=0x55d8c92f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f26feffd700 0 client.14970 ms_handle_reset on v2:172.21.15.179:6834/2178851321 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1892664559 shutdown_connections 2024-05-27T08:37:18.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.522+0000 7f2730aa2b80 1 -- 172.21.15.77:0/1892664559 wait complete. 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f2730aa2b80 20 client.14970 trim_cache size 0 max 16384 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24412, 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:37:18.530 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 116.274083029, 2024-05-27T08:37:18.531 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:37:18.531 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:37:18.531 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:37:18.531 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 162.87140846899999 2024-05-27T08:37:18.531 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:37:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 -- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27080b2880 msgr2=0x7f27080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27080b2880 0x7f27080b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f271000b580 tx=0x7f2710009f10 comp rx=0 tx=0).stop 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 -- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201041a0 msgr2=0x7f272019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201041a0 0x7f272019fcf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f271400c9b0 tx=0x7f271400ccc0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 -- 172.21.15.77:0/3045529687 shutdown_connections 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27080b2880 0x7f27080b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f27201088f0 0x7f27201999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f27201037f0 0x7f272019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 --2- 172.21.15.77:0/3045529687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27201041a0 0x7f272019fcf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 -- 172.21.15.77:0/3045529687 >> 172.21.15.77:0/3045529687 conn(0x7f272007a6a0 msgr2=0x7f2720101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 -- 172.21.15.77:0/3045529687 shutdown_connections 2024-05-27T08:37:18.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:18.523+0000 7f272620c700 1 -- 172.21.15.77:0/3045529687 wait complete. 2024-05-27T08:37:19.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:18 smithi179 ceph-mon[36481]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:19.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:18 smithi077 ceph-mon[32007]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:19.162 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-05-27T08:37:19.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:18 smithi092 ceph-mon[36049]: pgmap v355: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:19.404 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:20.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 -- 172.21.15.77:0/2092757528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 msgr2=0x7fc1101103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 --2- 172.21.15.77:0/2092757528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 0x7fc1101103d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc10800b0a0 tx=0x7fc10800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 -- 172.21.15.77:0/2092757528 shutdown_connections 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 --2- 172.21.15.77:0/2092757528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 0x7fc1101103d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 --2- 172.21.15.77:0/2092757528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1101095b0 0x7fc110109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 --2- 172.21.15.77:0/2092757528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc11006bc90 0x7fc11006c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 -- 172.21.15.77:0/2092757528 >> 172.21.15.77:0/2092757528 conn(0x7fc110079410 msgr2=0x7fc110079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 -- 172.21.15.77:0/2092757528 shutdown_connections 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 -- 172.21.15.77:0/2092757528 wait complete. 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 Processor -- start 2024-05-27T08:37:20.817 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.815+0000 7fc116be9700 1 -- start start 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc116be9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc11006bc90 0x7fc11019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc116be9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 0x7fc11019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc116be9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1101095b0 0x7fc110199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc116be9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc110113740 con 0x7fc11006c650 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc116be9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101135c0 con 0x7fc11006bc90 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc116be9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101138c0 con 0x7fc1101095b0 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 0x7fc11019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 0x7fc11019fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55246/0 (socket says 172.21.15.77:55246) 2024-05-27T08:37:20.819 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 -- 172.21.15.77:0/3370426102 learned_addr learned my addr 172.21.15.77:0/3370426102 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc115186700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1101095b0 0x7fc110199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc114985700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc11006bc90 0x7fc11019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 -- 172.21.15.77:0/3370426102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1101095b0 msgr2=0x7fc110199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1101095b0 0x7fc110199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 -- 172.21.15.77:0/3370426102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc11006bc90 msgr2=0x7fc11019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc11006bc90 0x7fc11019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 -- 172.21.15.77:0/3370426102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11019a1c0 con 0x7fc11006c650 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc114985700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc11006bc90 0x7fc11019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 0x7fc11019fd90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c920 tx=0x7fc0fc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10dffb700 1 -- 172.21.15.77:0/3370426102 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc007b00 con 0x7fc11006c650 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10dffb700 1 -- 172.21.15.77:0/3370426102 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0fc004ce0 con 0x7fc11006c650 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc10dffb700 1 -- 172.21.15.77:0/3370426102 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc005550 con 0x7fc11006c650 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.816+0000 7fc116be9700 1 -- 172.21.15.77:0/3370426102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11019a4b0 con 0x7fc11006c650 2024-05-27T08:37:20.820 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.817+0000 7fc116be9700 1 -- 172.21.15.77:0/3370426102 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1101a6720 con 0x7fc11006c650 2024-05-27T08:37:20.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.820+0000 7fc10dffb700 1 -- 172.21.15.77:0/3370426102 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc0fc007ca0 con 0x7fc11006c650 2024-05-27T08:37:20.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.820+0000 7fc10dffb700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1000b2900 0x7fc1000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.820+0000 7fc114985700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1000b2900 0x7fc1000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.820+0000 7fc10dffb700 1 -- 172.21.15.77:0/3370426102 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc0fc0e1fe0 con 0x7fc11006c650 2024-05-27T08:37:20.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc12147fb80 1 Processor -- start 2024-05-27T08:37:20.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc114985700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1000b2900 0x7fc1000b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc110100aa0 tx=0x7fc104008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:20.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc12147fb80 1 -- start start 2024-05-27T08:37:20.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b86c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b866cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 0x555f2b873030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc12147fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555f2b7c4d20 con 0x555f2b866850 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc12147fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555f2b7b48c0 con 0x555f2b867210 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc12147fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555f2b7b8900 con 0x555f2b86c2d0 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b866cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b866cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55248/0 (socket says 172.21.15.77:55248) 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc10ffff700 1 -- 172.21.15.77:0/3627444231 learned_addr learned my addr 172.21.15.77:0/3627444231 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc10ffff700 1 -- 172.21.15.77:0/3627444231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 msgr2=0x555f2b86c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3627444231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b86c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc10ffff700 1 -- 172.21.15.77:0/3627444231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 msgr2=0x555f2b873030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3627444231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 0x555f2b873030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.821+0000 7fc10ffff700 1 -- 172.21.15.77:0/3627444231 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2b75df60 con 0x555f2b866850 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3627444231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b866cd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0a5720 tx=0x7fc0fc0a9e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebff9ad20435c71 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc0faffd700 1 -- 172.21.15.77:0/3627444231 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc01d040 con 0x555f2b866850 2024-05-27T08:37:20.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc0faffd700 1 -- 172.21.15.77:0/3627444231 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0fc0b2500 con 0x555f2b866850 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc0faffd700 1 -- 172.21.15.77:0/3627444231 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc091210 con 0x555f2b866850 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc12147fb80 1 -- 172.21.15.77:0/3627444231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 msgr2=0x555f2b866cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3627444231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b866cd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0a5720 tx=0x7fc0fc0a9e20 comp rx=0 tx=0).stop 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc12147fb80 1 -- 172.21.15.77:0/3627444231 shutdown_connections 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3627444231 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 0x555f2b873030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3627444231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b86c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3627444231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b866cd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.822+0000 7fc12147fb80 1 -- 172.21.15.77:0/3627444231 >> 172.21.15.77:0/3627444231 conn(0x555f2b7921f0 msgr2=0x555f2b861390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 -- 172.21.15.77:0/3627444231 shutdown_connections 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 -- 172.21.15.77:0/3627444231 wait complete. 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 -- start start 2024-05-27T08:37:20.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 Processor -- start 2024-05-27T08:37:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b866850 0x555f2b872dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b867210 0x555f2b8700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b870610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555f2b6baf50 con 0x555f2b867210 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555f2b71cd50 con 0x555f2b866850 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555f2b873630 con 0x555f2b86c2d0 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.823+0000 7fc12147fb80 10 client.0 _ll_register_callbacks cb 0x555f2b74f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b867210 0x555f2b8700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b867210 0x555f2b8700d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55252/0 (socket says 172.21.15.77:55252) 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 -- 172.21.15.77:0/3527142194 learned_addr learned my addr 172.21.15.77:0/3527142194 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc115186700 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b870610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc114985700 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b866850 0x555f2b872dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 -- 172.21.15.77:0/3527142194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 msgr2=0x555f2b870610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b870610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 -- 172.21.15.77:0/3527142194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b866850 msgr2=0x555f2b872dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b866850 0x555f2b872dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 -- 172.21.15.77:0/3527142194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2b704880 con 0x555f2b867210 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc10ffff700 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b867210 0x555f2b8700d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc01a040 tx=0x7fc0fc00a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d190ee36bdce685a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc0effff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc0effff700 1 -- 172.21.15.77:0/3527142194 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc01d040 con 0x555f2b867210 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc0effff700 1 -- 172.21.15.77:0/3527142194 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc0fc0a9b10 con 0x555f2b867210 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc12147fb80 1 -- 172.21.15.77:0/3527142194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0fc0b2500 con 0x555f2b867210 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc0effff700 1 -- 172.21.15.77:0/3527142194 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc08a050 con 0x555f2b867210 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc12147fb80 10 client.14982 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc12147fb80 1 -- 172.21.15.77:0/3527142194 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f2b76e710 con 0x555f2b867210 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.824+0000 7fc0effff700 1 -- 172.21.15.77:0/3527142194 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0fc0b2500 con 0x555f2b867210 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc0effff700 1 -- 172.21.15.77:0/3527142194 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc0fc0a5010 con 0x555f2b867210 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc12147fb80 10 client.14982 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:20.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc12147fb80 10 client.14982 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc12147fb80 20 client.14982 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x555f2b87efa0 0x555f2b881070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc12147fb80 4 client.14982 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc12147fb80 1 -- 172.21.15.77:0/3527142194 --> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f2b7c6c70 con 0x555f2b87efa0 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc114985700 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x555f2b87efa0 0x555f2b881070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc0effff700 10 client.14982 ms_handle_connect on v2:172.21.15.179:6836/2857088493 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.825+0000 7fc114985700 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x555f2b87efa0 0x555f2b881070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4cb2356cdcdb8dc7 server_cookie=2a98970d65684165 in_seq=0 out_seq=0 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc0effff700 1 -- 172.21.15.77:0/3527142194 <== mds.2 v2:172.21.15.179:6836/2857088493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555f2b7c6c70 con 0x555f2b87efa0 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc0effff700 10 client.14982 handle_command_reply: tid=0 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc12147fb80 1 client.14982 shutdown 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc12147fb80 1 -- 172.21.15.77:0/3527142194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b867210 msgr2=0x555f2b8700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b867210 0x555f2b8700d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc01a040 tx=0x7fc0fc00a610 comp rx=0 tx=0).stop 2024-05-27T08:37:20.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc12147fb80 1 -- 172.21.15.77:0/3527142194 shutdown_connections 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x555f2b87efa0 0x555f2b881070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b870610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc0effff700 0 client.14982 ms_handle_reset on v2:172.21.15.179:6836/2857088493 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b866850 0x555f2b872dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.826+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3527142194 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b867210 0x555f2b8700d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.827+0000 7fc12147fb80 1 -- 172.21.15.77:0/3527142194 >> 172.21.15.77:0/3527142194 conn(0x555f2b7921f0 msgr2=0x555f2b860aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.827+0000 7fc12147fb80 1 -- 172.21.15.77:0/3527142194 shutdown_connections 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.827+0000 7fc12147fb80 1 -- 172.21.15.77:0/3527142194 wait complete. 2024-05-27T08:37:20.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.827+0000 7fc12147fb80 20 client.14982 trim_cache size 0 max 16384 2024-05-27T08:37:20.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.847+0000 7fc12147fb80 1 Processor -- start 2024-05-27T08:37:20.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.847+0000 7fc12147fb80 1 -- start start 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b86f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b867210 0x555f2b861220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b86c2d0 0x555f2b865f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc12147fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555f2b87f3b0 con 0x555f2b866850 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc12147fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555f2b87fa80 con 0x555f2b86c2d0 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc12147fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555f2b880150 con 0x555f2b867210 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b86f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b86f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55262/0 (socket says 172.21.15.77:55262) 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 -- 172.21.15.77:0/3464766979 learned_addr learned my addr 172.21.15.77:0/3464766979 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 -- 172.21.15.77:0/3464766979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b867210 msgr2=0x555f2b861220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 --2- 172.21.15.77:0/3464766979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b867210 0x555f2b861220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 -- 172.21.15.77:0/3464766979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b86c2d0 msgr2=0x555f2b865f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 --2- 172.21.15.77:0/3464766979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b86c2d0 0x555f2b865f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 -- 172.21.15.77:0/3464766979 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f2b86e640 con 0x555f2b866850 2024-05-27T08:37:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc114985700 1 --2- 172.21.15.77:0/3464766979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b86f250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc104007f20 tx=0x7fc10400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363bc2f7420bbcf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc0fa7fc700 1 -- 172.21.15.77:0/3464766979 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10402f070 con 0x555f2b866850 2024-05-27T08:37:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc0fa7fc700 1 -- 172.21.15.77:0/3464766979 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10400cd90 con 0x555f2b866850 2024-05-27T08:37:20.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.848+0000 7fc0fa7fc700 1 -- 172.21.15.77:0/3464766979 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104022640 con 0x555f2b866850 2024-05-27T08:37:20.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 -- 172.21.15.77:0/3464766979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 msgr2=0x555f2b86f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3464766979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b86f250 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc104007f20 tx=0x7fc10400f000 comp rx=0 tx=0).stop 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 -- 172.21.15.77:0/3464766979 shutdown_connections 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3464766979 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b86c2d0 0x555f2b865f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3464766979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b86f250 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 --2- 172.21.15.77:0/3464766979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b867210 0x555f2b861220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 -- 172.21.15.77:0/3464766979 >> 172.21.15.77:0/3464766979 conn(0x555f2b7921f0 msgr2=0x555f2b865840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 -- 172.21.15.77:0/3464766979 shutdown_connections 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 -- 172.21.15.77:0/3464766979 wait complete. 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 -- start start 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.849+0000 7fc12147fb80 1 Processor -- start 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b88a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.853 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 0x555f2b8878a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc12147fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b887de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc12147fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555f2b880ef0 con 0x555f2b866850 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc12147fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555f2b880820 con 0x555f2b867210 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc12147fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555f2b8815c0 con 0x555f2b86c2d0 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc12147fb80 10 client.0 _ll_register_callbacks cb 0x555f2b74f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b88a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b88a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55272/0 (socket says 172.21.15.77:55272) 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 -- 172.21.15.77:0/973163036 learned_addr learned my addr 172.21.15.77:0/973163036 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc10ffff700 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 0x555f2b8878a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc115186700 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b887de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 -- 172.21.15.77:0/973163036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 msgr2=0x555f2b887de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b887de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 -- 172.21.15.77:0/973163036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 msgr2=0x555f2b8878a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:20.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 0x555f2b8878a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 -- 172.21.15.77:0/973163036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f2b888320 con 0x555f2b866850 2024-05-27T08:37:20.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc114985700 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b88a8d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc10402d040 tx=0x7fc10402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6ae94360485c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:20.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.850+0000 7fc0effff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:20.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.853+0000 7fc0effff700 1 -- 172.21.15.77:0/973163036 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10402f040 con 0x555f2b866850 2024-05-27T08:37:20.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.853+0000 7fc0effff700 1 -- 172.21.15.77:0/973163036 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc10401c040 con 0x555f2b866850 2024-05-27T08:37:20.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.853+0000 7fc12147fb80 1 -- 172.21.15.77:0/973163036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10400cd90 con 0x555f2b866850 2024-05-27T08:37:20.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc0effff700 1 -- 172.21.15.77:0/973163036 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc10402a820 con 0x555f2b866850 2024-05-27T08:37:20.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc12147fb80 10 client.14988 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:20.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc12147fb80 1 -- 172.21.15.77:0/973163036 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f2b8f0120 con 0x555f2b866850 2024-05-27T08:37:20.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc0effff700 1 -- 172.21.15.77:0/973163036 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc10400cd90 con 0x555f2b866850 2024-05-27T08:37:20.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc0effff700 1 -- 172.21.15.77:0/973163036 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc104032030 con 0x555f2b866850 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc12147fb80 10 client.14988 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc12147fb80 10 client.14988 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc12147fb80 20 client.14988 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc12147fb80 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x555f2b87efa0 0x555f2b8f3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc12147fb80 4 client.14988 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc12147fb80 1 -- 172.21.15.77:0/973163036 --> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] -- command(tid 0: {"prefix": "status"}) -- 0x555f2b7c0450 con 0x555f2b87efa0 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.854+0000 7fc10ffff700 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x555f2b87efa0 0x555f2b8f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc0effff700 10 client.14988 ms_handle_connect on v2:172.21.15.179:6836/2857088493 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc10ffff700 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x555f2b87efa0 0x555f2b8f3490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=581cd93cbb82639d server_cookie=ac62da3d6a434c4b in_seq=0 out_seq=0 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc0effff700 1 -- 172.21.15.77:0/973163036 <== mds.2 v2:172.21.15.179:6836/2857088493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555f2b7c0450 con 0x555f2b87efa0 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc0effff700 10 client.14988 handle_command_reply: tid=0 2024-05-27T08:37:20.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 client.14988 shutdown 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 -- 172.21.15.77:0/973163036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 msgr2=0x555f2b88a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b88a8d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc10402d040 tx=0x7fc10402a730 comp rx=0 tx=0).stop 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 -- 172.21.15.77:0/973163036 shutdown_connections 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x555f2b87efa0 0x555f2b8f3490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f2b86c2d0 0x555f2b887de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc0effff700 0 client.14988 ms_handle_reset on v2:172.21.15.179:6836/2857088493 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555f2b866850 0x555f2b88a8d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 --2- 172.21.15.77:0/973163036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555f2b867210 0x555f2b8878a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 -- 172.21.15.77:0/973163036 >> 172.21.15.77:0/973163036 conn(0x555f2b7921f0 msgr2=0x555f2b871900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 -- 172.21.15.77:0/973163036 shutdown_connections 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 1 -- 172.21.15.77:0/973163036 wait complete. 2024-05-27T08:37:20.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.855+0000 7fc12147fb80 20 client.14988 trim_cache size 0 max 16384 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24418, 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 119.611848006, 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:37:20.860 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 162.78586595100001 2024-05-27T08:37:20.861 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:37:20.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 -- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1000b2900 msgr2=0x7fc1000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1000b2900 0x7fc1000b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc110100aa0 tx=0x7fc104008040 comp rx=0 tx=0).stop 2024-05-27T08:37:20.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 -- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 msgr2=0x7fc11019fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:20.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 0x7fc11019fd90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c920 tx=0x7fc0fc00cc30 comp rx=0 tx=0).stop 2024-05-27T08:37:20.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 -- 172.21.15.77:0/3370426102 shutdown_connections 2024-05-27T08:37:20.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc1000b2900 0x7fc1000b4dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1101095b0 0x7fc110199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc11006bc90 0x7fc11019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 --2- 172.21.15.77:0/3370426102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc11006c650 0x7fc11019fd90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:20.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 -- 172.21.15.77:0/3370426102 >> 172.21.15.77:0/3370426102 conn(0x7fc110079410 msgr2=0x7fc1101011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:20.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 -- 172.21.15.77:0/3370426102 shutdown_connections 2024-05-27T08:37:20.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:20.856+0000 7fc116be9700 1 -- 172.21.15.77:0/3370426102 wait complete. 2024-05-27T08:37:21.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:20 smithi179 ceph-mon[36481]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:21.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:20 smithi077 ceph-mon[32007]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:21.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:20 smithi092 ceph-mon[36049]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:21.545 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-05-27T08:37:21.789 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:23.088 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:22 smithi077 ceph-mon[32007]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:37:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:22 smithi179 ceph-mon[36481]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:37:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:22 smithi092 ceph-mon[36049]: pgmap v357: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.340+0000 7faeb309f700 1 -- 172.21.15.77:0/1131445190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 msgr2=0x7faeac10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.340+0000 7faeb309f700 1 --2- 172.21.15.77:0/1131445190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 0x7faeac10f2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faea400b0a0 tx=0x7faea400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.340+0000 7faeb309f700 1 -- 172.21.15.77:0/1131445190 shutdown_connections 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.340+0000 7faeb309f700 1 --2- 172.21.15.77:0/1131445190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 0x7faeac10f2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.340+0000 7faeb309f700 1 --2- 172.21.15.77:0/1131445190 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeac1084b0 0x7faeac108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.340+0000 7faeb309f700 1 --2- 172.21.15.77:0/1131445190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeac102470 0x7faeac1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.340+0000 7faeb309f700 1 -- 172.21.15.77:0/1131445190 >> 172.21.15.77:0/1131445190 conn(0x7faeac079410 msgr2=0x7faeac079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.341+0000 7faeb309f700 1 -- 172.21.15.77:0/1131445190 shutdown_connections 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.341+0000 7faeb309f700 1 -- 172.21.15.77:0/1131445190 wait complete. 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.341+0000 7faeb309f700 1 Processor -- start 2024-05-27T08:37:23.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.341+0000 7faeb309f700 1 -- start start 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb309f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeac102470 0x7faeac19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb309f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 0x7faeac19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb309f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeac1084b0 0x7faeac1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb309f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faeac1125c0 con 0x7faeac102e30 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb309f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faeac112440 con 0x7faeac1084b0 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb309f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faeac112740 con 0x7faeac102470 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 0x7faeac19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 0x7faeac19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55286/0 (socket says 172.21.15.77:55286) 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 -- 172.21.15.77:0/2808639725 learned_addr learned my addr 172.21.15.77:0/2808639725 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb163c700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeac1084b0 0x7faeac1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeac102470 0x7faeac19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 -- 172.21.15.77:0/2808639725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeac102470 msgr2=0x7faeac19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeac102470 0x7faeac19f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 -- 172.21.15.77:0/2808639725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeac1084b0 msgr2=0x7faeac1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeac1084b0 0x7faeac1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 -- 172.21.15.77:0/2808639725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac19a150 con 0x7faeac102e30 2024-05-27T08:37:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeabfff700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 0x7faeac19fd70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c920 tx=0x7fae9800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faea9ffb700 1 -- 172.21.15.77:0/2808639725 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98004a30 con 0x7faeac102e30 2024-05-27T08:37:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faeb309f700 1 -- 172.21.15.77:0/2808639725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac19a440 con 0x7faeac102e30 2024-05-27T08:37:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.342+0000 7faea9ffb700 1 -- 172.21.15.77:0/2808639725 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae980054d0 con 0x7faeac102e30 2024-05-27T08:37:23.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.343+0000 7faea9ffb700 1 -- 172.21.15.77:0/2808639725 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98007bd0 con 0x7faeac102e30 2024-05-27T08:37:23.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.343+0000 7faeb309f700 1 -- 172.21.15.77:0/2808639725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac1a6700 con 0x7faeac102e30 2024-05-27T08:37:23.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.345+0000 7faea9ffb700 1 -- 172.21.15.77:0/2808639725 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae98004bd0 con 0x7faeac102e30 2024-05-27T08:37:23.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.346+0000 7faea9ffb700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fae9c0b2900 0x7fae9c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.346+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fae9c0b2900 0x7fae9c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.346+0000 7faea9ffb700 1 -- 172.21.15.77:0/2808639725 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fae980e1cf0 con 0x7faeac102e30 2024-05-27T08:37:23.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.346+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fae9c0b2900 0x7fae9c0b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faeac19ab30 tx=0x7faea000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:23.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faebd935b80 1 Processor -- start 2024-05-27T08:37:23.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faebd935b80 1 -- start start 2024-05-27T08:37:23.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faebd935b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faebd935b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 0x55ff0ebeecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faebd935b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faebd935b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff0eb4cd20 con 0x55ff0ebef210 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faebd935b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ff0eb3c8c0 con 0x55ff0ebf42d0 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faebd935b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ff0eb40900 con 0x55ff0ebee850 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb163c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb163c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55298/0 (socket says 172.21.15.77:55298) 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb163c700 1 -- 172.21.15.77:0/2392847039 learned_addr learned my addr 172.21.15.77:0/2392847039 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeabfff700 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 0x55ff0ebeecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb163c700 1 -- 172.21.15.77:0/2392847039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 msgr2=0x55ff0ebeecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb163c700 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 0x55ff0ebeecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb163c700 1 -- 172.21.15.77:0/2392847039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 msgr2=0x55ff0ebf46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb163c700 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.347+0000 7faeb163c700 1 -- 172.21.15.77:0/2392847039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0eae5f60 con 0x55ff0ebef210 2024-05-27T08:37:23.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7faeb163c700 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfb030 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faea4009230 tx=0x7faea4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7dbca949e4a9f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:23.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7fae96ffd700 1 -- 172.21.15.77:0/2392847039 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4020070 con 0x55ff0ebef210 2024-05-27T08:37:23.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7fae96ffd700 1 -- 172.21.15.77:0/2392847039 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea4003c00 con 0x55ff0ebef210 2024-05-27T08:37:23.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7fae96ffd700 1 -- 172.21.15.77:0/2392847039 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4005040 con 0x55ff0ebef210 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7faebd935b80 1 -- 172.21.15.77:0/2392847039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 msgr2=0x55ff0ebfb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7faebd935b80 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfb030 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faea4009230 tx=0x7faea4009260 comp rx=0 tx=0).stop 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7faebd935b80 1 -- 172.21.15.77:0/2392847039 shutdown_connections 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7faebd935b80 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfb030 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7faebd935b80 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.348+0000 7faebd935b80 1 --2- 172.21.15.77:0/2392847039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 0x55ff0ebeecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 -- 172.21.15.77:0/2392847039 >> 172.21.15.77:0/2392847039 conn(0x55ff0eb1a1f0 msgr2=0x55ff0ebe9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 -- 172.21.15.77:0/2392847039 shutdown_connections 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 -- 172.21.15.77:0/2392847039 wait complete. 2024-05-27T08:37:23.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 -- start start 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 Processor -- start 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 0x55ff0ebfac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebef210 0x55ff0ebf7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff0ea42f50 con 0x55ff0ebf42d0 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ff0eaa4d50 con 0x55ff0ebee850 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.349+0000 7faebd935b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ff0ebffa40 con 0x55ff0ebef210 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faebd935b80 10 client.0 _ll_register_callbacks cb 0x55ff0ead7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55306/0 (socket says 172.21.15.77:55306) 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 -- 172.21.15.77:0/2863442472 learned_addr learned my addr 172.21.15.77:0/2863442472 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 0x55ff0ebfac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.356 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeabfff700 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebef210 0x55ff0ebf7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 -- 172.21.15.77:0/2863442472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebef210 msgr2=0x55ff0ebf7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebef210 0x55ff0ebf7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 -- 172.21.15.77:0/2863442472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 msgr2=0x55ff0ebfac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 0x55ff0ebfac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 -- 172.21.15.77:0/2863442472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff0ea8c880 con 0x55ff0ebf42d0 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faeb163c700 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf8520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faea400b3f0 tx=0x7faea4009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e8ea7e2755ab40 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7fae8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7fae8bfff700 1 -- 172.21.15.77:0/2863442472 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4020040 con 0x55ff0ebf42d0 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7fae8bfff700 1 -- 172.21.15.77:0/2863442472 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7faea40050c0 con 0x55ff0ebf42d0 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faebd935b80 1 -- 172.21.15.77:0/2863442472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea4003c00 con 0x55ff0ebf42d0 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7fae8bfff700 1 -- 172.21.15.77:0/2863442472 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faea4007970 con 0x55ff0ebf42d0 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.350+0000 7faebd935b80 10 client.15000 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:23.357 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7faebd935b80 1 -- 172.21.15.77:0/2863442472 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff0eaf6710 con 0x55ff0ebf42d0 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7fae8bfff700 1 -- 172.21.15.77:0/2863442472 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea4003c00 con 0x55ff0ebf42d0 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7fae8bfff700 1 -- 172.21.15.77:0/2863442472 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faea4011070 con 0x55ff0ebf42d0 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7faebd935b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7faebd935b80 10 client.15000 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7faebd935b80 20 client.15000 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7faebd935b80 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55ff0ec0b260 0x55ff0ec0d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7faebd935b80 4 client.15000 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7faebd935b80 1 -- 172.21.15.77:0/2863442472 --> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff0eb4ec70 con 0x55ff0ec0b260 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.351+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55ff0ec0b260 0x55ff0ec0d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.352+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55ff0ec0b260 0x55ff0ec0d3f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca6589cc76e015d server_cookie=2d991a3386aa9f75 in_seq=0 out_seq=0 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.352+0000 7fae8bfff700 10 client.15000 ms_handle_connect on v2:172.21.15.179:6838/2672999131 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7fae8bfff700 1 -- 172.21.15.77:0/2863442472 <== mds.1 v2:172.21.15.179:6838/2672999131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ff0eb4ec70 con 0x55ff0ec0b260 2024-05-27T08:37:23.358 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7fae8bfff700 10 client.15000 handle_command_reply: tid=0 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 client.15000 shutdown 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 -- 172.21.15.77:0/2863442472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebf42d0 msgr2=0x55ff0ebf8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf8520 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faea400b3f0 tx=0x7faea4009550 comp rx=0 tx=0).stop 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 -- 172.21.15.77:0/2863442472 shutdown_connections 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55ff0ec0b260 0x55ff0ec0d3f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf8520 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 0x55ff0ebfac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 --2- 172.21.15.77:0/2863442472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebef210 0x55ff0ebf7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 -- 172.21.15.77:0/2863442472 >> 172.21.15.77:0/2863442472 conn(0x55ff0eb1a1f0 msgr2=0x55ff0ebe8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7fae8bfff700 0 client.15000 ms_handle_reset on v2:172.21.15.179:6838/2672999131 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 -- 172.21.15.77:0/2863442472 shutdown_connections 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 1 -- 172.21.15.77:0/2863442472 wait complete. 2024-05-27T08:37:23.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.353+0000 7faebd935b80 20 client.15000 trim_cache size 0 max 16384 2024-05-27T08:37:23.376 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.374+0000 7faebd935b80 1 Processor -- start 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.374+0000 7faebd935b80 1 -- start start 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.374+0000 7faebd935b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 0x55ff0ebe9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.374+0000 7faebd935b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.374+0000 7faebd935b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.374+0000 7faebd935b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff0ec0b730 con 0x55ff0ebef210 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faebd935b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ff0ec0be00 con 0x55ff0ebf42d0 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faebd935b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ff0ec0c4d0 con 0x55ff0ebee850 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebff590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55314/0 (socket says 172.21.15.77:55314) 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 -- 172.21.15.77:0/2906114793 learned_addr learned my addr 172.21.15.77:0/2906114793 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeb163c700 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 -- 172.21.15.77:0/2906114793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 msgr2=0x55ff0ebe9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:23.377 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 0x55ff0ebe9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 0x55ff0ebe9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 -- 172.21.15.77:0/2906114793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 msgr2=0x55ff0ebf97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:23.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 -- 172.21.15.77:0/2906114793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0ec0d760 con 0x55ff0ebef210 2024-05-27T08:37:23.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7faeabfff700 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebff590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fae9801e000 tx=0x7fae980a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d7919c07d6bd81 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:23.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7fae967fc700 1 -- 172.21.15.77:0/2906114793 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9801d040 con 0x55ff0ebef210 2024-05-27T08:37:23.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7fae967fc700 1 -- 172.21.15.77:0/2906114793 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae980a9310 con 0x55ff0ebef210 2024-05-27T08:37:23.378 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.375+0000 7fae967fc700 1 -- 172.21.15.77:0/2906114793 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980a9570 con 0x55ff0ebef210 2024-05-27T08:37:23.381 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 -- 172.21.15.77:0/2906114793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 msgr2=0x55ff0ebff590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebff590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fae9801e000 tx=0x7fae980a8c60 comp rx=0 tx=0).stop 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 -- 172.21.15.77:0/2906114793 shutdown_connections 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebf97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebee850 0x55ff0ebe9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 --2- 172.21.15.77:0/2906114793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebff590 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 -- 172.21.15.77:0/2906114793 >> 172.21.15.77:0/2906114793 conn(0x55ff0eb1a1f0 msgr2=0x55ff0ebee0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 -- 172.21.15.77:0/2906114793 shutdown_connections 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 -- 172.21.15.77:0/2906114793 wait complete. 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 -- start start 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.376+0000 7faebd935b80 1 Processor -- start 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faebd935b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 0x55ff0ebfb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.382 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faebd935b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faebd935b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebfbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faebd935b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ff0ec0d270 con 0x55ff0ebef210 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faebd935b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55ff0ec0cba0 con 0x55ff0ebee850 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faebd935b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ff0ec0d940 con 0x55ff0ebf42d0 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faebd935b80 10 client.0 _ll_register_callbacks cb 0x55ff0ead7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55320/0 (socket says 172.21.15.77:55320) 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 -- 172.21.15.77:0/501869786 learned_addr learned my addr 172.21.15.77:0/501869786 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 0x55ff0ebfb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeb163c700 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebfbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 -- 172.21.15.77:0/501869786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebf42d0 msgr2=0x55ff0ebfbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebfbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 -- 172.21.15.77:0/501869786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 msgr2=0x55ff0ebfb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:23.383 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 0x55ff0ebfb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 -- 172.21.15.77:0/501869786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff0ec16b00 con 0x55ff0ebef210 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faeabfff700 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfbab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fae980a9010 tx=0x7fae98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cead62b677a79fa server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7fae8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7fae8bfff700 1 -- 172.21.15.77:0/501869786 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980c3070 con 0x55ff0ebef210 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7fae8bfff700 1 -- 172.21.15.77:0/501869786 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fae980a9780 con 0x55ff0ebef210 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.377+0000 7faebd935b80 1 -- 172.21.15.77:0/501869786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae980a9310 con 0x55ff0ebef210 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7fae8bfff700 1 -- 172.21.15.77:0/501869786 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fae9801d040 con 0x55ff0ebef210 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faebd935b80 10 client.15006 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faebd935b80 1 -- 172.21.15.77:0/501869786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ff0ec7c760 con 0x55ff0ebef210 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7fae8bfff700 1 -- 172.21.15.77:0/501869786 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae980a9310 con 0x55ff0ebef210 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7fae8bfff700 1 -- 172.21.15.77:0/501869786 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fae980ca0c0 con 0x55ff0ebef210 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faebd935b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faebd935b80 10 client.15006 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T08:37:23.384 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faebd935b80 20 client.15006 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faebd935b80 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55ff0ec80dd0 0x55ff0ec7fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faebd935b80 4 client.15006 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faebd935b80 1 -- 172.21.15.77:0/501869786 --> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] -- command(tid 0: {"prefix": "status"}) -- 0x55ff0eb48450 con 0x55ff0ec80dd0 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.378+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55ff0ec80dd0 0x55ff0ec7fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7fae8bfff700 10 client.15006 ms_handle_connect on v2:172.21.15.179:6838/2672999131 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faeb0e3b700 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55ff0ec80dd0 0x55ff0ec7fae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b130d6e1b61644a0 server_cookie=525e3e9c0949482e in_seq=0 out_seq=0 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7fae8bfff700 1 -- 172.21.15.77:0/501869786 <== mds.1 v2:172.21.15.179:6838/2672999131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ff0eb48450 con 0x55ff0ec80dd0 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7fae8bfff700 10 client.15006 handle_command_reply: tid=0 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 client.15006 shutdown 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 -- 172.21.15.77:0/501869786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 msgr2=0x55ff0ebfbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfbab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fae980a9010 tx=0x7fae98004750 comp rx=0 tx=0).stop 2024-05-27T08:37:23.385 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 -- 172.21.15.77:0/501869786 shutdown_connections 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55ff0ec80dd0 0x55ff0ec7fae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff0ebf42d0 0x55ff0ebfbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55ff0ebee850 0x55ff0ebfb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 --2- 172.21.15.77:0/501869786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ff0ebef210 0x55ff0ebfbab0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 -- 172.21.15.77:0/501869786 >> 172.21.15.77:0/501869786 conn(0x55ff0eb1a1f0 msgr2=0x55ff0ebf1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7fae8bfff700 0 client.15006 ms_handle_reset on v2:172.21.15.179:6838/2672999131 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 -- 172.21.15.77:0/501869786 shutdown_connections 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 1 -- 172.21.15.77:0/501869786 wait complete. 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.379+0000 7faebd935b80 20 client.15006 trim_cache size 0 max 16384 2024-05-27T08:37:23.386 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24433, 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 123.14803629799999, 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 159.96482169199999 2024-05-27T08:37:23.387 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:37:23.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 -- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fae9c0b2900 msgr2=0x7fae9c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fae9c0b2900 0x7fae9c0b4dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faeac19ab30 tx=0x7faea000b410 comp rx=0 tx=0).stop 2024-05-27T08:37:23.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 -- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 msgr2=0x7faeac19fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:23.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 0x7faeac19fd70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c920 tx=0x7fae9800cc30 comp rx=0 tx=0).stop 2024-05-27T08:37:23.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 -- 172.21.15.77:0/2808639725 shutdown_connections 2024-05-27T08:37:23.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faeac1084b0 0x7faeac1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fae9c0b2900 0x7fae9c0b4dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.388 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faeac102470 0x7faeac19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 --2- 172.21.15.77:0/2808639725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faeac102e30 0x7faeac19fd70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:23.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 -- 172.21.15.77:0/2808639725 >> 172.21.15.77:0/2808639725 conn(0x7faeac079410 msgr2=0x7faeac06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:23.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 -- 172.21.15.77:0/2808639725 shutdown_connections 2024-05-27T08:37:23.389 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:23.380+0000 7faeb309f700 1 -- 172.21.15.77:0/2808639725 wait complete. 2024-05-27T08:37:24.020 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T08:37:24.261 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:25.034 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:24 smithi077 ceph-mon[32007]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:25.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:24 smithi092 ceph-mon[36049]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:25.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:24 smithi179 ceph-mon[36481]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:25.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 -- 172.21.15.77:0/2626237667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101037f0 msgr2=0x7f2810103bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 --2- 172.21.15.77:0/2626237667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101037f0 0x7f2810103bd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f280c0099c0 tx=0x7f280c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 -- 172.21.15.77:0/2626237667 shutdown_connections 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 --2- 172.21.15.77:0/2626237667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28101088f0 0x7f2810110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 --2- 172.21.15.77:0/2626237667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101037f0 0x7f2810103bd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 --2- 172.21.15.77:0/2626237667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28101041a0 0x7f28101081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 -- 172.21.15.77:0/2626237667 >> 172.21.15.77:0/2626237667 conn(0x7f281007a6a0 msgr2=0x7f281007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 -- 172.21.15.77:0/2626237667 shutdown_connections 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 -- 172.21.15.77:0/2626237667 wait complete. 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 Processor -- start 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.724+0000 7f2817424700 1 -- start start 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f2817424700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28101037f0 0x7f281019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f2817424700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101041a0 0x7f281019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f2817424700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28101088f0 0x7f2810199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f2817424700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2810113730 con 0x7f28101041a0 2024-05-27T08:37:25.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f2817424700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f28101135b0 con 0x7f28101037f0 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f2817424700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28101138b0 con 0x7f28101088f0 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101041a0 0x7f281019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101041a0 0x7f281019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55338/0 (socket says 172.21.15.77:55338) 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 -- 172.21.15.77:0/574347554 learned_addr learned my addr 172.21.15.77:0/574347554 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28159c1700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28101088f0 0x7f2810199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28151c0700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28101037f0 0x7f281019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 -- 172.21.15.77:0/574347554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28101088f0 msgr2=0x7f2810199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28101088f0 0x7f2810199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 -- 172.21.15.77:0/574347554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28101037f0 msgr2=0x7f281019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.729 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28101037f0 0x7f281019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 -- 172.21.15.77:0/574347554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281019a1d0 con 0x7f28101041a0 2024-05-27T08:37:25.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28151c0700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28101037f0 0x7f281019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:25.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28149bf700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101041a0 0x7f281019fda0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f280000b530 tx=0x7f280000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:25.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.725+0000 7f28067fc700 1 -- 172.21.15.77:0/574347554 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800015020 con 0x7f28101041a0 2024-05-27T08:37:25.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.726+0000 7f28067fc700 1 -- 172.21.15.77:0/574347554 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2800011ce0 con 0x7f28101041a0 2024-05-27T08:37:25.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.726+0000 7f2817424700 1 -- 172.21.15.77:0/574347554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281019a4c0 con 0x7f28101041a0 2024-05-27T08:37:25.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.726+0000 7f28067fc700 1 -- 172.21.15.77:0/574347554 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800010e40 con 0x7f28101041a0 2024-05-27T08:37:25.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.726+0000 7f2817424700 1 -- 172.21.15.77:0/574347554 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28101a6730 con 0x7f28101041a0 2024-05-27T08:37:25.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.728+0000 7f28067fc700 1 -- 172.21.15.77:0/574347554 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2800018020 con 0x7f28101041a0 2024-05-27T08:37:25.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.729+0000 7f28067fc700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27fc0b2880 0x7f27fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.732 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.729+0000 7f28151c0700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27fc0b2880 0x7f27fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.729+0000 7f28067fc700 1 -- 172.21.15.77:0/574347554 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f28000e1a80 con 0x7f28101041a0 2024-05-27T08:37:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27fc0b2880 0x7f27fc0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c00 tx=0x7f280c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f2821cbab80 1 Processor -- start 2024-05-27T08:37:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f2821cbab80 1 -- start start 2024-05-27T08:37:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9af12d0 0x5595d9af16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 0x5595d9aebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.733 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aec210 0x5595d9af8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f2821cbab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5595d9a49d20 con 0x5595d9af12d0 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f2821cbab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595d9a398c0 con 0x5595d9aeb850 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f2821cbab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5595d9a3d900 con 0x5595d9aec210 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9af12d0 0x5595d9af16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9af12d0 0x5595d9af16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55350/0 (socket says 172.21.15.77:55350) 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 -- 172.21.15.77:0/1214452605 learned_addr learned my addr 172.21.15.77:0/1214452605 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28149bf700 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 0x5595d9aebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28159c1700 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aec210 0x5595d9af8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 -- 172.21.15.77:0/1214452605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aec210 msgr2=0x5595d9af8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:25.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aec210 0x5595d9af8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 -- 172.21.15.77:0/1214452605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 msgr2=0x5595d9aebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 0x5595d9aebcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 -- 172.21.15.77:0/1214452605 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d99e2f60 con 0x5595d9af12d0 2024-05-27T08:37:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.730+0000 7f28151c0700 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9af12d0 0x5595d9af16b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f280c015000 tx=0x7f280c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7c1fd8d42ea73e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f27fb7fe700 1 -- 172.21.15.77:0/1214452605 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c003920 con 0x5595d9af12d0 2024-05-27T08:37:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f27fb7fe700 1 -- 172.21.15.77:0/1214452605 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f280c005030 con 0x5595d9af12d0 2024-05-27T08:37:25.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f27fb7fe700 1 -- 172.21.15.77:0/1214452605 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c02f660 con 0x5595d9af12d0 2024-05-27T08:37:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f2821cbab80 1 -- 172.21.15.77:0/1214452605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9af12d0 msgr2=0x5595d9af16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9af12d0 0x5595d9af16b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f280c015000 tx=0x7f280c01f000 comp rx=0 tx=0).stop 2024-05-27T08:37:25.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f2821cbab80 1 -- 172.21.15.77:0/1214452605 shutdown_connections 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aec210 0x5595d9af8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9af12d0 0x5595d9af16b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1214452605 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 0x5595d9aebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.731+0000 7f2821cbab80 1 -- 172.21.15.77:0/1214452605 >> 172.21.15.77:0/1214452605 conn(0x5595d9a171f0 msgr2=0x5595d9ae6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 -- 172.21.15.77:0/1214452605 shutdown_connections 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 -- 172.21.15.77:0/1214452605 wait complete. 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 -- start start 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 Processor -- start 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 0x5595d9af7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9af4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9af5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5595d993ff50 con 0x5595d9aec210 2024-05-27T08:37:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595d99a1d50 con 0x5595d9aeb850 2024-05-27T08:37:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.732+0000 7f2821cbab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5595d9af8630 con 0x5595d9af12d0 2024-05-27T08:37:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f2821cbab80 10 client.0 _ll_register_callbacks cb 0x5595d99d4900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9af4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9af4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55360/0 (socket says 172.21.15.77:55360) 2024-05-27T08:37:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 -- 172.21.15.77:0/1298136057 learned_addr learned my addr 172.21.15.77:0/1298136057 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:25.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28151c0700 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 0x5595d9af7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28159c1700 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9af5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 -- 172.21.15.77:0/1298136057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 msgr2=0x5595d9af5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9af5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 -- 172.21.15.77:0/1298136057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 msgr2=0x5595d9af7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 0x5595d9af7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 -- 172.21.15.77:0/1298136057 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d9989880 con 0x5595d9aec210 2024-05-27T08:37:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28159c1700 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9af5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:25.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f28149bf700 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9af4f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f280000b500 tx=0x7f28000a8c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db6b59855be4a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:25.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f27f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:25.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1298136057 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28000ad410 con 0x5595d9aec210 2024-05-27T08:37:25.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1298136057 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f28000a5650 con 0x5595d9aec210 2024-05-27T08:37:25.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f2821cbab80 1 -- 172.21.15.77:0/1298136057 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280c005030 con 0x5595d9aec210 2024-05-27T08:37:25.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1298136057 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28000b1ec0 con 0x5595d9aec210 2024-05-27T08:37:25.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f2821cbab80 10 client.15018 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:25.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.733+0000 7f2821cbab80 1 -- 172.21.15.77:0/1298136057 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595d99f3710 con 0x5595d9aec210 2024-05-27T08:37:25.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1298136057 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f280c005030 con 0x5595d9aec210 2024-05-27T08:37:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1298136057 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f28000a49d0 con 0x5595d9aec210 2024-05-27T08:37:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f2821cbab80 10 client.15018 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f2821cbab80 10 client.15018 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T08:37:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f2821cbab80 20 client.15018 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5595d9b050e0 0x5595d9b06550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f2821cbab80 4 client.15018 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:37:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f2821cbab80 1 -- 172.21.15.77:0/1298136057 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595d9a4bc70 con 0x5595d9b050e0 2024-05-27T08:37:25.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f28151c0700 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5595d9b050e0 0x5595d9b06550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f27f8ff9700 10 client.15018 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:37:25.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.734+0000 7f28151c0700 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5595d9b050e0 0x5595d9b06550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e98fd18d1bd17b78 server_cookie=ef5ff2634ac8c2f0 in_seq=0 out_seq=0 2024-05-27T08:37:25.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1298136057 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595d9a4bc70 con 0x5595d9b050e0 2024-05-27T08:37:25.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f27f8ff9700 10 client.15018 handle_command_reply: tid=0 2024-05-27T08:37:25.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 client.15018 shutdown 2024-05-27T08:37:25.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 -- 172.21.15.77:0/1298136057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 msgr2=0x5595d9af4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9af4f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f280000b500 tx=0x7f28000a8c20 comp rx=0 tx=0).stop 2024-05-27T08:37:25.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 -- 172.21.15.77:0/1298136057 shutdown_connections 2024-05-27T08:37:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5595d9b050e0 0x5595d9b06550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9af5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aeb850 0x5595d9af7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1298136057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9af4f50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f2821cbab80 1 -- 172.21.15.77:0/1298136057 >> 172.21.15.77:0/1298136057 conn(0x5595d9a171f0 msgr2=0x5595d9ae5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.735+0000 7f27f8ff9700 0 client.15018 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:37:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.736+0000 7f2821cbab80 1 -- 172.21.15.77:0/1298136057 shutdown_connections 2024-05-27T08:37:25.746 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.736+0000 7f2821cbab80 1 -- 172.21.15.77:0/1298136057 wait complete. 2024-05-27T08:37:25.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.736+0000 7f2821cbab80 20 client.15018 trim_cache size 0 max 16384 2024-05-27T08:37:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f2821cbab80 1 Processor -- start 2024-05-27T08:37:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f2821cbab80 1 -- start start 2024-05-27T08:37:25.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aeb850 0x5595d9af0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9aef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9af12d0 0x5595d9ae6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f2821cbab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5595d9b04890 con 0x5595d9aec210 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f2821cbab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595d9b04f60 con 0x5595d9af12d0 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f2821cbab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5595d9b05630 con 0x5595d9aeb850 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f28149bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9aef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f28149bf700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9aef3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55368/0 (socket says 172.21.15.77:55368) 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.756+0000 7f28149bf700 1 -- 172.21.15.77:0/2343472409 learned_addr learned my addr 172.21.15.77:0/2343472409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f28149bf700 1 -- 172.21.15.77:0/2343472409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aeb850 msgr2=0x5595d9af0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f28149bf700 1 --2- 172.21.15.77:0/2343472409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aeb850 0x5595d9af0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f28149bf700 1 -- 172.21.15.77:0/2343472409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9af12d0 msgr2=0x5595d9ae6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f28149bf700 1 --2- 172.21.15.77:0/2343472409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9af12d0 0x5595d9ae6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.760 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f28149bf700 1 -- 172.21.15.77:0/2343472409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d9af0ef0 con 0x5595d9aec210 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f28149bf700 1 --2- 172.21.15.77:0/2343472409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9aef3a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f28000a8d60 tx=0x7f28000ad2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67edcac8b9ac756 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f27faffd700 1 -- 172.21.15.77:0/2343472409 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28000a49d0 con 0x5595d9aec210 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f27faffd700 1 -- 172.21.15.77:0/2343472409 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28000a4b70 con 0x5595d9aec210 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.757+0000 7f27faffd700 1 -- 172.21.15.77:0/2343472409 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28000a7d80 con 0x5595d9aec210 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 -- 172.21.15.77:0/2343472409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 msgr2=0x5595d9aef3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 --2- 172.21.15.77:0/2343472409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9aef3a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f28000a8d60 tx=0x7f28000ad2e0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 -- 172.21.15.77:0/2343472409 shutdown_connections 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 --2- 172.21.15.77:0/2343472409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9af12d0 0x5595d9ae6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 --2- 172.21.15.77:0/2343472409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9aeb850 0x5595d9af0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 --2- 172.21.15.77:0/2343472409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aec210 0x5595d9aef3a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 -- 172.21.15.77:0/2343472409 >> 172.21.15.77:0/2343472409 conn(0x5595d9a171f0 msgr2=0x5595d9aeec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 -- 172.21.15.77:0/2343472409 shutdown_connections 2024-05-27T08:37:25.761 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 -- 172.21.15.77:0/2343472409 wait complete. 2024-05-27T08:37:25.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 -- start start 2024-05-27T08:37:25.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.758+0000 7f2821cbab80 1 Processor -- start 2024-05-27T08:37:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aeb850 0x5595d9b0ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aec210 0x5595d9b0f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f2821cbab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9b0f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f2821cbab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5595d9b063d0 con 0x5595d9aeb850 2024-05-27T08:37:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f2821cbab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5595d9b05d00 con 0x5595d9aec210 2024-05-27T08:37:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f2821cbab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5595d9b06aa0 con 0x5595d9af12d0 2024-05-27T08:37:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f2821cbab80 10 client.0 _ll_register_callbacks cb 0x5595d99d4900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:25.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aeb850 0x5595d9b0ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aeb850 0x5595d9b0ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55380/0 (socket says 172.21.15.77:55380) 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 -- 172.21.15.77:0/1230833442 learned_addr learned my addr 172.21.15.77:0/1230833442 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28159c1700 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9b0f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28149bf700 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aec210 0x5595d9b0f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 -- 172.21.15.77:0/1230833442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 msgr2=0x5595d9b0f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9b0f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 -- 172.21.15.77:0/1230833442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aec210 msgr2=0x5595d9b0f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aec210 0x5595d9b0f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 -- 172.21.15.77:0/1230833442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d9b749b0 con 0x5595d9aeb850 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f28151c0700 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aeb850 0x5595d9b0ed90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f280c009750 tx=0x7f280c0059c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3657339854d2473 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f27f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:25.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1230833442 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c005070 con 0x5595d9aeb850 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1230833442 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f280c0055f0 con 0x5595d9aeb850 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.759+0000 7f2821cbab80 1 -- 172.21.15.77:0/1230833442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28000a4b70 con 0x5595d9aeb850 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1230833442 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f280c003920 con 0x5595d9aeb850 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f2821cbab80 10 client.15024 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f2821cbab80 1 -- 172.21.15.77:0/1230833442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595d9b75c30 con 0x5595d9aeb850 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1230833442 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28000a4b70 con 0x5595d9aeb850 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1230833442 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f280c02fc20 con 0x5595d9aeb850 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f2821cbab80 10 client.15024 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f2821cbab80 10 client.15024 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f2821cbab80 20 client.15024 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5595d9b7a230 0x5595d9b78f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f2821cbab80 4 client.15024 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:37:25.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f2821cbab80 1 -- 172.21.15.77:0/1230833442 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "status"}) -- 0x5595d9a45450 con 0x5595d9b7a230 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.760+0000 7f28149bf700 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5595d9b7a230 0x5595d9b78f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f27f8ff9700 10 client.15024 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f28149bf700 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5595d9b7a230 0x5595d9b78f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d8ad3d9f48b2127 server_cookie=fd217b34cdbd921b in_seq=0 out_seq=0 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f27f8ff9700 1 -- 172.21.15.77:0/1230833442 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5595d9a45450 con 0x5595d9b7a230 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f27f8ff9700 10 client.15024 handle_command_reply: tid=0 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 client.15024 shutdown 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 -- 172.21.15.77:0/1230833442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aeb850 msgr2=0x5595d9b0ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aeb850 0x5595d9b0ed90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f280c009750 tx=0x7f280c0059c0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 -- 172.21.15.77:0/1230833442 shutdown_connections 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5595d9b7a230 0x5595d9b78f40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595d9af12d0 0x5595d9b0f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f27f8ff9700 0 client.15024 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5595d9aeb850 0x5595d9b0ed90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 --2- 172.21.15.77:0/1230833442 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5595d9aec210 0x5595d9b0f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 -- 172.21.15.77:0/1230833442 >> 172.21.15.77:0/1230833442 conn(0x5595d9a171f0 msgr2=0x5595d9aeec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 -- 172.21.15.77:0/1230833442 shutdown_connections 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 1 -- 172.21.15.77:0/1230833442 wait complete. 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.761+0000 7f2821cbab80 20 client.15024 trim_cache size 0 max 16384 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24434, 2024-05-27T08:37:25.769 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:37:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:37:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:37:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 136.72032362600001, 2024-05-27T08:37:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:37:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:37:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:37:25.770 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 152.10730564100001 2024-05-27T08:37:25.770 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 -- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27fc0b2880 msgr2=0x7f27fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27fc0b2880 0x7f27fc0b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c00 tx=0x7f280c005fb0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 -- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101041a0 msgr2=0x7f281019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101041a0 0x7f281019fda0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f280000b530 tx=0x7f280000b840 comp rx=0 tx=0).stop 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 -- 172.21.15.77:0/574347554 shutdown_connections 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f27fc0b2880 0x7f27fc0b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28101088f0 0x7f2810199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28101037f0 0x7f281019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 --2- 172.21.15.77:0/574347554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28101041a0 0x7f281019fda0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:25.771 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 -- 172.21.15.77:0/574347554 >> 172.21.15.77:0/574347554 conn(0x7f281007a6a0 msgr2=0x7f2810109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 -- 172.21.15.77:0/574347554 shutdown_connections 2024-05-27T08:37:25.772 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:25.762+0000 7f2817424700 1 -- 172.21.15.77:0/574347554 wait complete. 2024-05-27T08:37:26.437 INFO:tasks.cephadm:Setup complete, yielding 2024-05-27T08:37:26.437 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T08:37:26.448 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:37:26.448 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-05-27T08:37:26.688 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:27.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:26 smithi077 ceph-mon[32007]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:26 smithi092 ceph-mon[36049]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:26 smithi179 ceph-mon[36481]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:28.255 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 -- 172.21.15.77:0/235635782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 msgr2=0x7f67a0108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 --2- 172.21.15.77:0/235635782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 0x7f67a0108d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6794009990 tx=0x7f6794009ca0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 -- 172.21.15.77:0/235635782 shutdown_connections 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 --2- 172.21.15.77:0/235635782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67a0109290 0x7f67a010b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 --2- 172.21.15.77:0/235635782 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a0104030 0x7f67a0106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 --2- 172.21.15.77:0/235635782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 0x7f67a0108d50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 -- 172.21.15.77:0/235635782 >> 172.21.15.77:0/235635782 conn(0x7f67a00fda40 msgr2=0x7f67a00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 -- 172.21.15.77:0/235635782 shutdown_connections 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 -- 172.21.15.77:0/235635782 wait complete. 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 Processor -- start 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.252+0000 7f67a5a45700 1 -- start start 2024-05-27T08:37:28.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a5a45700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67a0104030 0x7f67a0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a5a45700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 0x7f67a0101fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a5a45700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a0109290 0x7f67a0102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a5a45700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67a010f060 con 0x7f67a0106960 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a5a45700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f67a010eee0 con 0x7f67a0109290 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a5a45700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67a010f1e0 con 0x7f67a0104030 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 0x7f67a0101fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 0x7f67a0101fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:55400/0 (socket says 172.21.15.77:55400) 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 -- 172.21.15.77:0/2482318064 learned_addr learned my addr 172.21.15.77:0/2482318064 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679effd700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67a0104030 0x7f67a0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679f7fe700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a0109290 0x7f67a0102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 -- 172.21.15.77:0/2482318064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67a0104030 msgr2=0x7f67a0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67a0104030 0x7f67a0103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 -- 172.21.15.77:0/2482318064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a0109290 msgr2=0x7f67a0102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a0109290 0x7f67a0102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 -- 172.21.15.77:0/2482318064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a0102dd0 con 0x7f67a0106960 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679f7fe700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a0109290 0x7f67a0102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f679e7fc700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 0x7f67a0101fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67940098e0 tx=0x7f67940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a4a43700 1 -- 172.21.15.77:0/2482318064 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794026070 con 0x7f67a0106960 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a4a43700 1 -- 172.21.15.77:0/2482318064 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f679400fbc0 con 0x7f67a0106960 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a01a3820 con 0x7f67a0106960 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.253+0000 7f67a4a43700 1 -- 172.21.15.77:0/2482318064 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794017650 con 0x7f67a0106960 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.254+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a01a3c70 con 0x7f67a0106960 2024-05-27T08:37:28.258 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.254+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a006a230 con 0x7f67a0106960 2024-05-27T08:37:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.256+0000 7f67a4a43700 1 -- 172.21.15.77:0/2482318064 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6794029070 con 0x7f67a0106960 2024-05-27T08:37:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.256+0000 7f67a4a43700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f67880b2880 0x7f67880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.257+0000 7f679effd700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f67880b2880 0x7f67880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.257+0000 7f67a4a43700 1 -- 172.21.15.77:0/2482318064 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f67940e3730 con 0x7f67a0106960 2024-05-27T08:37:28.259 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.257+0000 7f679effd700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f67880b2880 0x7f67880b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f679000bbd0 tx=0x7f679000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:28.261 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.260+0000 7f67a4a43700 1 -- 172.21.15.77:0/2482318064 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6794030070 con 0x7f67a0106960 2024-05-27T08:37:28.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.485+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f67a0063a40 con 0x7f67880b2880 2024-05-27T08:37:28.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.486+0000 7f67a4a43700 1 -- 172.21.15.77:0/2482318064 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f67a0063a40 con 0x7f67880b2880 2024-05-27T08:37:28.487 INFO:teuthology.orchestra.run.smithi077.stdout:Backend: cephadm 2024-05-27T08:37:28.487 INFO:teuthology.orchestra.run.smithi077.stdout:Available: Yes 2024-05-27T08:37:28.488 INFO:teuthology.orchestra.run.smithi077.stdout:Paused: No 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f67880b2880 msgr2=0x7f67880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f67880b2880 0x7f67880b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f679000bbd0 tx=0x7f679000b5b0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 msgr2=0x7f67a0101fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 0x7f67a0101fe0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67940098e0 tx=0x7f67940047b0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 shutdown_connections 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f67a0109290 0x7f67a0102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f67880b2880 0x7f67880b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67a0104030 0x7f67a0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 --2- 172.21.15.77:0/2482318064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67a0106960 0x7f67a0101fe0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:28.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.488+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 >> 172.21.15.77:0/2482318064 conn(0x7f67a00fda40 msgr2=0x7f67a0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:28.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.489+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 shutdown_connections 2024-05-27T08:37:28.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:28.489+0000 7f67a5a45700 1 -- 172.21.15.77:0/2482318064 wait complete. 2024-05-27T08:37:29.048 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:28 smithi077 ceph-mon[32007]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:29.074 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-27T08:37:29.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:28 smithi092 ceph-mon[36049]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:29.325 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:29.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:28 smithi179 ceph-mon[36481]: pgmap v360: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:30.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:29 smithi077 ceph-mon[32007]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:30.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:29 smithi092 ceph-mon[36049]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:30.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:29 smithi179 ceph-mon[36481]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:30.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.859+0000 7f7529a41700 1 -- 172.21.15.77:0/1196255922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 msgr2=0x7f752410a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.859+0000 7f7529a41700 1 --2- 172.21.15.77:0/1196255922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 0x7f752410a8f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f751c00b0d0 tx=0x7f751c00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.859+0000 7f7529a41700 1 -- 172.21.15.77:0/1196255922 shutdown_connections 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.859+0000 7f7529a41700 1 --2- 172.21.15.77:0/1196255922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 0x7f752410a8f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.859+0000 7f7529a41700 1 --2- 172.21.15.77:0/1196255922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75241024a0 0x7f75241028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.860+0000 7f7529a41700 1 --2- 172.21.15.77:0/1196255922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75241036a0 0x7f7524103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.860+0000 7f7529a41700 1 -- 172.21.15.77:0/1196255922 >> 172.21.15.77:0/1196255922 conn(0x7f75240fda40 msgr2=0x7f75240ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.860+0000 7f7529a41700 1 -- 172.21.15.77:0/1196255922 shutdown_connections 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.860+0000 7f7529a41700 1 -- 172.21.15.77:0/1196255922 wait complete. 2024-05-27T08:37:30.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.860+0000 7f7529a41700 1 Processor -- start 2024-05-27T08:37:30.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.860+0000 7f7529a41700 1 -- start start 2024-05-27T08:37:30.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7529a41700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75241024a0 0x7f7524198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:30.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7529a41700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75241036a0 0x7f7524198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:30.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7529a41700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 0x7f752419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:30.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7529a41700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f752410df90 con 0x7f7524104040 2024-05-27T08:37:30.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7529a41700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f752410de10 con 0x7f75241036a0 2024-05-27T08:37:30.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7529a41700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f752410e110 con 0x7f75241024a0 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 0x7f752419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 0x7f752419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37896/0 (socket says 172.21.15.77:37896) 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 -- 172.21.15.77:0/3468016914 learned_addr learned my addr 172.21.15.77:0/3468016914 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7522ffd700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75241024a0 0x7f7524198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75227fc700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75241036a0 0x7f7524198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 -- 172.21.15.77:0/3468016914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75241024a0 msgr2=0x7f7524198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75241024a0 0x7f7524198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 -- 172.21.15.77:0/3468016914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75241036a0 msgr2=0x7f7524198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75241036a0 0x7f7524198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 -- 172.21.15.77:0/3468016914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752406b220 con 0x7f7524104040 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75227fc700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75241036a0 0x7f7524198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:30.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f75237fe700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 0x7f752419fc00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f751c0079d0 tx=0x7f751c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:30.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7528a3f700 1 -- 172.21.15.77:0/3468016914 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c021070 con 0x7f7524104040 2024-05-27T08:37:30.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.861+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752406b4b0 con 0x7f7524104040 2024-05-27T08:37:30.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.862+0000 7f7528a3f700 1 -- 172.21.15.77:0/3468016914 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f751c01db00 con 0x7f7524104040 2024-05-27T08:37:30.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.862+0000 7f7528a3f700 1 -- 172.21.15.77:0/3468016914 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c01bab0 con 0x7f7524104040 2024-05-27T08:37:30.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.862+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752406ba40 con 0x7f7524104040 2024-05-27T08:37:30.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.862+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752406a230 con 0x7f7524104040 2024-05-27T08:37:30.867 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.864+0000 7f7528a3f700 1 -- 172.21.15.77:0/3468016914 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f751c024070 con 0x7f7524104040 2024-05-27T08:37:30.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.864+0000 7f7528a3f700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f750c0b2880 0x7f750c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:30.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.864+0000 7f7522ffd700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f750c0b2880 0x7f750c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:30.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.865+0000 7f7528a3f700 1 -- 172.21.15.77:0/3468016914 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f751c00f020 con 0x7f7524104040 2024-05-27T08:37:30.868 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.865+0000 7f7522ffd700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f750c0b2880 0x7f750c0b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7524103500 tx=0x7f751400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:30.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:30.868+0000 7f7528a3f700 1 -- 172.21.15.77:0/3468016914 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f751c02b020 con 0x7f7524104040 2024-05-27T08:37:31.090 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.088+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f752406bc10 con 0x7f750c0b2880 2024-05-27T08:37:31.090 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:30 smithi077 ceph-mon[32007]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:31.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.102+0000 7f7528a3f700 1 -- 172.21.15.77:0/3468016914 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f752406bc10 con 0x7f750c0b2880 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:mds.a smithi077 running (2m) 43s ago 2m 16.6M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 f89bb5c0f009 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:mds.b smithi092 running (2m) 86s ago 2m 26.3M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 e737fd95726e 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:mds.c smithi179 running (2m) 86s ago 2m 17.7M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 de3049cdd643 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:mds.d smithi077 running (3m) 43s ago 3m 16.6M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 161d1c4d41cd 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:mds.e smithi092 running (2m) 86s ago 2m 19.5M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 e417bc2498d9 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:mds.f smithi179 running (2m) 86s ago 2m 17.4M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 0b34f7b54725 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:mds.g smithi077 running (2m) 43s ago 2m 15.8M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 37f65f013fbc 2024-05-27T08:37:31.104 INFO:teuthology.orchestra.run.smithi077.stdout:mds.h smithi092 running (2m) 86s ago 2m 17.5M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 2ec7d89d4004 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mds.i smithi179 running (2m) 86s ago 2m 20.4M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 b660e7f6b09b 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mds.j smithi077 running (3m) 43s ago 3m 16.5M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 1d9ce26d82a5 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mds.k smithi092 running (3m) 86s ago 3m 18.0M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 d6c6c8fda2d8 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mds.l smithi179 running (3m) 86s ago 3m 15.5M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 4746f09de9fa 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.x smithi077 *:9283,8765 running (10m) 43s ago 10m 529M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 2deab0fd4f35 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.y smithi092 *:8443,8765 running (8m) 86s ago 8m 468M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 25224e5b15df 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mgr.z smithi179 *:8443,8765 running (8m) 86s ago 8m 469M - 19.0.0-3868-ga9cb3a58 47fc2147e0a3 2d5719555a42 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mon.a smithi077 running (10m) 43s ago 10m 49.7M 2048M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 59b893a78295 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mon.b smithi092 running (8m) 86s ago 8m 39.2M 2048M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 7d3aeeafd63c 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:mon.c smithi179 running (8m) 86s ago 8m 43.2M 2048M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 04b807590ee2 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 smithi077 running (8m) 43s ago 8m 49.2M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 40a8730d1f8f 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 smithi092 running (7m) 86s ago 7m 57.5M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 8ba154e834df 2024-05-27T08:37:31.105 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 smithi179 running (7m) 86s ago 7m 51.5M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 55bb1ca76dd1 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 smithi077 running (6m) 43s ago 6m 58.3M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 63e635a2bd11 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 smithi092 running (6m) 86s ago 6m 45.6M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 b243d4ef621d 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 smithi179 running (6m) 86s ago 6m 57.7M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 2ab3d9f9b8d7 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 smithi077 running (5m) 43s ago 5m 51.5M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 a1a78094f407 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 smithi092 running (5m) 86s ago 5m 45.8M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 9614a9e301a8 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.8 smithi179 running (4m) 86s ago 4m 46.2M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 5e04a1d44554 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.9 smithi077 running (4m) 43s ago 4m 52.6M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 3aadfe1dd744 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.10 smithi092 running (3m) 86s ago 3m 55.0M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 8bc1eb7f780e 2024-05-27T08:37:31.106 INFO:teuthology.orchestra.run.smithi077.stdout:osd.11 smithi179 running (3m) 86s ago 3m 43.1M 4272M 19.0.0-3868-ga9cb3a58 47fc2147e0a3 dcb7b5949639 2024-05-27T08:37:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.105+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f750c0b2880 msgr2=0x7f750c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.105+0000 7f7529a41700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f750c0b2880 0x7f750c0b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7524103500 tx=0x7f751400a380 comp rx=0 tx=0).stop 2024-05-27T08:37:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.105+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 msgr2=0x7f752419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.105+0000 7f7529a41700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 0x7f752419fc00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f751c0079d0 tx=0x7f751c007a00 comp rx=0 tx=0).stop 2024-05-27T08:37:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.105+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 shutdown_connections 2024-05-27T08:37:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.106+0000 7f7529a41700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7524104040 0x7f752419fc00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.106+0000 7f7529a41700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f750c0b2880 0x7f750c0b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:31.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.106+0000 7f7529a41700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75241024a0 0x7f7524198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.106+0000 7f7529a41700 1 --2- 172.21.15.77:0/3468016914 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f75241036a0 0x7f7524198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.106+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 >> 172.21.15.77:0/3468016914 conn(0x7f75240fda40 msgr2=0x7f75241069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.106+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 shutdown_connections 2024-05-27T08:37:31.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:31.106+0000 7f7529a41700 1 -- 172.21.15.77:0/3468016914 wait complete. 2024-05-27T08:37:31.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:30 smithi092 ceph-mon[36049]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:31.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:30 smithi179 ceph-mon[36481]: pgmap v361: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:31.757 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-27T08:37:31.999 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:33.078 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:32 smithi077 ceph-mon[32007]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:33.079 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:32 smithi077 ceph-mon[32007]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:37:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:32 smithi092 ceph-mon[36049]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:32 smithi092 ceph-mon[36049]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:37:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:32 smithi179 ceph-mon[36481]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:32 smithi179 ceph-mon[36481]: pgmap v362: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:37:33.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 -- 172.21.15.77:0/1091605503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e01035c0 msgr2=0x7ff8e01039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:33.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/1091605503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e01035c0 0x7ff8e01039e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00099c0 tx=0x7ff8d0009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 -- 172.21.15.77:0/1091605503 shutdown_connections 2024-05-27T08:37:33.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/1091605503 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e0105160 0x7ff8e010ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/1091605503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e01035c0 0x7ff8e01039e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/1091605503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8e01047c0 0x7ff8e0104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 -- 172.21.15.77:0/1091605503 >> 172.21.15.77:0/1091605503 conn(0x7ff8e00feb60 msgr2=0x7ff8e0100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 -- 172.21.15.77:0/1091605503 shutdown_connections 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.639+0000 7ff8e62ab700 1 -- 172.21.15.77:0/1091605503 wait complete. 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e62ab700 1 Processor -- start 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e62ab700 1 -- start start 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e62ab700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8e01035c0 0x7ff8e019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e62ab700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e01047c0 0x7ff8e019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e62ab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e0105160 0x7ff8e01a2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e62ab700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e010f120 con 0x7ff8e0105160 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e62ab700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e010efa0 con 0x7ff8e01047c0 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e62ab700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e010f2a0 con 0x7ff8e01035c0 2024-05-27T08:37:33.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e4848700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e0105160 0x7ff8e01a2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e4848700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e0105160 0x7ff8e01a2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37908/0 (socket says 172.21.15.77:37908) 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e4848700 1 -- 172.21.15.77:0/8028184 learned_addr learned my addr 172.21.15.77:0/8028184 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8dffff700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8e01035c0 0x7ff8e019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8d7fff700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e01047c0 0x7ff8e019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e4848700 1 -- 172.21.15.77:0/8028184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8e01035c0 msgr2=0x7ff8e019b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e4848700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8e01035c0 0x7ff8e019b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e4848700 1 -- 172.21.15.77:0/8028184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e01047c0 msgr2=0x7ff8e019bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e4848700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e01047c0 0x7ff8e019bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.640+0000 7ff8e4848700 1 -- 172.21.15.77:0/8028184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e01a3640 con 0x7ff8e0105160 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.641+0000 7ff8e4848700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e0105160 0x7ff8e01a2fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800b9a0 tx=0x7ff8d800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.641+0000 7ff8ddffb700 1 -- 172.21.15.77:0/8028184 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8019070 con 0x7ff8e0105160 2024-05-27T08:37:33.646 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.641+0000 7ff8ddffb700 1 -- 172.21.15.77:0/8028184 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8d800cde0 con 0x7ff8e0105160 2024-05-27T08:37:33.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.641+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e01a3930 con 0x7ff8e0105160 2024-05-27T08:37:33.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.641+0000 7ff8ddffb700 1 -- 172.21.15.77:0/8028184 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8007370 con 0x7ff8e0105160 2024-05-27T08:37:33.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.641+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01a3e20 con 0x7ff8e0105160 2024-05-27T08:37:33.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.642+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8e006a230 con 0x7ff8e0105160 2024-05-27T08:37:33.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.643+0000 7ff8ddffb700 1 -- 172.21.15.77:0/8028184 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff8d801c070 con 0x7ff8e0105160 2024-05-27T08:37:33.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.644+0000 7ff8ddffb700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff8cc0b2880 0x7ff8cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:33.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.644+0000 7ff8dffff700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff8cc0b2880 0x7ff8cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:33.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.644+0000 7ff8ddffb700 1 -- 172.21.15.77:0/8028184 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ff8d80e17c0 con 0x7ff8e0105160 2024-05-27T08:37:33.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.644+0000 7ff8dffff700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff8cc0b2880 0x7ff8cc0b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009f50 tx=0x7ff8d000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:33.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.649+0000 7ff8ddffb700 1 -- 172.21.15.77:0/8028184 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8d8051df0 con 0x7ff8e0105160 2024-05-27T08:37:33.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.861+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff8e0076e90 con 0x7ff8cc0b2880 2024-05-27T08:37:33.869 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.867+0000 7ff8ddffb700 1 -- 172.21.15.77:0/8028184 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff8e0076e90 con 0x7ff8cc0b2880 2024-05-27T08:37:33.870 INFO:teuthology.orchestra.run.smithi077.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-27T08:37:33.870 INFO:teuthology.orchestra.run.smithi077.stdout:mds.all 12/12 89s ago 3m smithi077=a;smithi077=d;smithi077=g;smithi077=j;smithi092=b;smithi092=e;smithi092=h;smithi092=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-05-27T08:37:33.870 INFO:teuthology.orchestra.run.smithi077.stdout:mgr 3/3 89s ago 8m smithi077=x;smithi092=y;smithi179=z;count:3 2024-05-27T08:37:33.870 INFO:teuthology.orchestra.run.smithi077.stdout:mon 3/3 89s ago 8m smithi077:172.21.15.77=a;smithi092:172.21.15.92=b;smithi179:172.21.15.179=c;count:3 2024-05-27T08:37:33.870 INFO:teuthology.orchestra.run.smithi077.stdout:osd 12 89s ago - 2024-05-27T08:37:33.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff8cc0b2880 msgr2=0x7ff8cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:33.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff8cc0b2880 0x7ff8cc0b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009f50 tx=0x7ff8d000b540 comp rx=0 tx=0).stop 2024-05-27T08:37:33.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e0105160 msgr2=0x7ff8e01a2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:33.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e0105160 0x7ff8e01a2fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800b9a0 tx=0x7ff8d800bcb0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 shutdown_connections 2024-05-27T08:37:33.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8e0105160 0x7ff8e01a2fa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ff8cc0b2880 0x7ff8cc0b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.872 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8e01035c0 0x7ff8e019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 --2- 172.21.15.77:0/8028184 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff8e01047c0 0x7ff8e019bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:33.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.869+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 >> 172.21.15.77:0/8028184 conn(0x7ff8e00feb60 msgr2=0x7ff8e0075770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:33.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.870+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 shutdown_connections 2024-05-27T08:37:33.873 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:33.870+0000 7ff8e62ab700 1 -- 172.21.15.77:0/8028184 wait complete. 2024-05-27T08:37:34.504 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-27T08:37:34.744 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:35.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:34 smithi077 ceph-mon[32007]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:35.122 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:34 smithi077 ceph-mon[32007]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:35.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:34 smithi092 ceph-mon[36049]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:35.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:34 smithi092 ceph-mon[36049]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:34 smithi179 ceph-mon[36481]: pgmap v363: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:34 smithi179 ceph-mon[36481]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:36.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 -- 172.21.15.77:0/1149690892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1081015f0 msgr2=0x7fc108101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:36.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 --2- 172.21.15.77:0/1149690892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1081015f0 0x7fc108101a50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009990 tx=0x7fc0fc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 -- 172.21.15.77:0/1149690892 shutdown_connections 2024-05-27T08:37:36.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 --2- 172.21.15.77:0/1149690892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc108101f90 0x7fc10810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.361 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 --2- 172.21.15.77:0/1149690892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc108104b70 0x7fc108104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 --2- 172.21.15.77:0/1149690892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1081015f0 0x7fc108101a50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 -- 172.21.15.77:0/1149690892 >> 172.21.15.77:0/1149690892 conn(0x7fc1080fd1d0 msgr2=0x7fc1080ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 -- 172.21.15.77:0/1149690892 shutdown_connections 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 -- 172.21.15.77:0/1149690892 wait complete. 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.357+0000 7fc10de08700 1 Processor -- start 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc10de08700 1 -- start start 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc10de08700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1081015f0 0x7fc108199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc10de08700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc108101f90 0x7fc108199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc10de08700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc108104b70 0x7fc1081a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc10de08700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc10810f120 con 0x7fc108101f90 2024-05-27T08:37:36.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc10de08700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc10810efa0 con 0x7fc1081015f0 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc10de08700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc10810f2a0 con 0x7fc108104b70 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc108101f90 0x7fc108199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc108101f90 0x7fc108199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37934/0 (socket says 172.21.15.77:37934) 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 -- 172.21.15.77:0/3307716055 learned_addr learned my addr 172.21.15.77:0/3307716055 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc1077fe700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1081015f0 0x7fc108199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc107fff700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc108104b70 0x7fc1081a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 -- 172.21.15.77:0/3307716055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc108104b70 msgr2=0x7fc1081a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc108104b70 0x7fc1081a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 -- 172.21.15.77:0/3307716055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1081015f0 msgr2=0x7fc108199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1081015f0 0x7fc108199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 -- 172.21.15.77:0/3307716055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1081a13f0 con 0x7fc108101f90 2024-05-27T08:37:36.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.358+0000 7fc106ffd700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc108101f90 0x7fc108199cd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0059d0 tx=0x7fc0fc003890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:36.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.359+0000 7fc104ff9700 1 -- 172.21.15.77:0/3307716055 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc027070 con 0x7fc108101f90 2024-05-27T08:37:36.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.359+0000 7fc104ff9700 1 -- 172.21.15.77:0/3307716055 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0fc0041f0 con 0x7fc108101f90 2024-05-27T08:37:36.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.359+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1081a1680 con 0x7fc108101f90 2024-05-27T08:37:36.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.359+0000 7fc104ff9700 1 -- 172.21.15.77:0/3307716055 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00f460 con 0x7fc108101f90 2024-05-27T08:37:36.364 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.359+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1081a1b50 con 0x7fc108101f90 2024-05-27T08:37:36.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.360+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10806a230 con 0x7fc108101f90 2024-05-27T08:37:36.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.361+0000 7fc104ff9700 1 -- 172.21.15.77:0/3307716055 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc0fc02a070 con 0x7fc108101f90 2024-05-27T08:37:36.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.362+0000 7fc104ff9700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc0f00b2880 0x7fc0f00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:36.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.362+0000 7fc1077fe700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc0f00b2880 0x7fc0f00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:36.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.362+0000 7fc104ff9700 1 -- 172.21.15.77:0/3307716055 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc0fc0e33b0 con 0x7fc108101f90 2024-05-27T08:37:36.365 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.362+0000 7fc1077fe700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc0f00b2880 0x7fc0f00b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc10806af40 tx=0x7fc0f8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:36.367 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.365+0000 7fc104ff9700 1 -- 172.21.15.77:0/3307716055 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc0fc013070 con 0x7fc108101f90 2024-05-27T08:37:36.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.576+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc108063980 con 0x7fc0f00b2880 2024-05-27T08:37:36.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.577+0000 7fc104ff9700 1 -- 172.21.15.77:0/3307716055 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc108063980 con 0x7fc0f00b2880 2024-05-27T08:37:36.580 INFO:teuthology.orchestra.run.smithi077.stdout:HOST ADDR LABELS STATUS 2024-05-27T08:37:36.580 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 172.21.15.77 2024-05-27T08:37:36.581 INFO:teuthology.orchestra.run.smithi077.stdout:smithi092 172.21.15.92 2024-05-27T08:37:36.581 INFO:teuthology.orchestra.run.smithi077.stdout:smithi179 172.21.15.179 2024-05-27T08:37:36.581 INFO:teuthology.orchestra.run.smithi077.stdout:3 hosts in cluster 2024-05-27T08:37:36.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc0f00b2880 msgr2=0x7fc0f00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:36.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc0f00b2880 0x7fc0f00b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc10806af40 tx=0x7fc0f8009190 comp rx=0 tx=0).stop 2024-05-27T08:37:36.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc108101f90 msgr2=0x7fc108199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:36.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc108101f90 0x7fc108199cd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0059d0 tx=0x7fc0fc003890 comp rx=0 tx=0).stop 2024-05-27T08:37:36.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 shutdown_connections 2024-05-27T08:37:36.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc0f00b2880 0x7fc0f00b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc108104b70 0x7fc1081a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc1081015f0 0x7fc108199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 --2- 172.21.15.77:0/3307716055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc108101f90 0x7fc108199cd0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:36.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 >> 172.21.15.77:0/3307716055 conn(0x7fc1080fd1d0 msgr2=0x7fc108107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:36.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 shutdown_connections 2024-05-27T08:37:36.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:36.580+0000 7fc10de08700 1 -- 172.21.15.77:0/3307716055 wait complete. 2024-05-27T08:37:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:36 smithi077 ceph-mon[32007]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:37.204 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-27T08:37:37.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:36 smithi092 ceph-mon[36049]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:37.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:36 smithi179 ceph-mon[36481]: pgmap v364: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:37.446 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:38.149 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:37 smithi077 ceph-mon[32007]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:38.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:37 smithi092 ceph-mon[36049]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:38.257 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:37 smithi179 ceph-mon[36481]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 -- 172.21.15.77:0/34145832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c1024a0 msgr2=0x7f3a4c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 --2- 172.21.15.77:0/34145832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c1024a0 0x7f3a4c1028c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0099c0 tx=0x7f3a3c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 -- 172.21.15.77:0/34145832 shutdown_connections 2024-05-27T08:37:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 --2- 172.21.15.77:0/34145832 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c104040 0x7f3a4c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 --2- 172.21.15.77:0/34145832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c1024a0 0x7f3a4c1028c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 --2- 172.21.15.77:0/34145832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c1036a0 0x7f3a4c103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 -- 172.21.15.77:0/34145832 >> 172.21.15.77:0/34145832 conn(0x7f3a4c0fda40 msgr2=0x7f3a4c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 -- 172.21.15.77:0/34145832 shutdown_connections 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.929+0000 7f3a51f18700 1 -- 172.21.15.77:0/34145832 wait complete. 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a51f18700 1 Processor -- start 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a51f18700 1 -- start start 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a51f18700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c1024a0 0x7f3a4c06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a51f18700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c1036a0 0x7f3a4c1a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a51f18700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c104040 0x7f3a4c1a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a51f18700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c10dfc0 con 0x7f3a4c104040 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a51f18700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c10de40 con 0x7f3a4c1036a0 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a51f18700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c10e140 con 0x7f3a4c1024a0 2024-05-27T08:37:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c104040 0x7f3a4c1a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a4bfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c104040 0x7f3a4c1a2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37952/0 (socket says 172.21.15.77:37952) 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a4bfff700 1 -- 172.21.15.77:0/3938322360 learned_addr learned my addr 172.21.15.77:0/3938322360 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a4b7fe700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c1024a0 0x7f3a4c06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.930+0000 7f3a43fff700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c1036a0 0x7f3a4c1a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a4bfff700 1 -- 172.21.15.77:0/3938322360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c1024a0 msgr2=0x7f3a4c06bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a4bfff700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c1024a0 0x7f3a4c06bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a4bfff700 1 -- 172.21.15.77:0/3938322360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c1036a0 msgr2=0x7f3a4c1a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a4bfff700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c1036a0 0x7f3a4c1a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a4bfff700 1 -- 172.21.15.77:0/3938322360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c1a3650 con 0x7f3a4c104040 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a43fff700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c1036a0 0x7f3a4c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a4bfff700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c104040 0x7f3a4c1a2e70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b970 tx=0x7f3a4400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a497fa700 1 -- 172.21.15.77:0/3938322360 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44019070 con 0x7f3a4c104040 2024-05-27T08:37:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c1a3940 con 0x7f3a4c104040 2024-05-27T08:37:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a497fa700 1 -- 172.21.15.77:0/3938322360 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a4400cde0 con 0x7f3a4c104040 2024-05-27T08:37:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a497fa700 1 -- 172.21.15.77:0/3938322360 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44007370 con 0x7f3a4c104040 2024-05-27T08:37:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.931+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c1a3e30 con 0x7f3a4c104040 2024-05-27T08:37:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.932+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4c06a230 con 0x7f3a4c104040 2024-05-27T08:37:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.933+0000 7f3a497fa700 1 -- 172.21.15.77:0/3938322360 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3a4401c070 con 0x7f3a4c104040 2024-05-27T08:37:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.934+0000 7f3a497fa700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a380b2880 0x7f3a380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.934+0000 7f3a4b7fe700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a380b2880 0x7f3a380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.934+0000 7f3a497fa700 1 -- 172.21.15.77:0/3938322360 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3a440e1e20 con 0x7f3a4c104040 2024-05-27T08:37:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.934+0000 7f3a4b7fe700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a380b2880 0x7f3a380b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c006010 tx=0x7f3a3c005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:38.937+0000 7f3a497fa700 1 -- 172.21.15.77:0/3938322360 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a44023070 con 0x7f3a4c104040 2024-05-27T08:37:39.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:38 smithi077 ceph-mon[32007]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.157+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f3a4c063980 con 0x7f3a380b2880 2024-05-27T08:37:39.162 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.159+0000 7f3a497fa700 1 -- 172.21.15.77:0/3938322360 <== mgr.14152 v2:172.21.15.77:6800/2118880698 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f3a4c063980 con 0x7f3a380b2880 2024-05-27T08:37:39.162 INFO:teuthology.orchestra.run.smithi077.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-27T08:37:39.163 INFO:teuthology.orchestra.run.smithi077.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-27T08:37:39.163 INFO:teuthology.orchestra.run.smithi077.stdout:smithi092 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QM400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-27T08:37:39.163 INFO:teuthology.orchestra.run.smithi077.stdout:smithi179 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.161+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a380b2880 msgr2=0x7f3a380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.161+0000 7f3a51f18700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a380b2880 0x7f3a380b4d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c006010 tx=0x7f3a3c005c00 comp rx=0 tx=0).stop 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.161+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c104040 msgr2=0x7f3a4c1a2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.161+0000 7f3a51f18700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c104040 0x7f3a4c1a2e70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400b970 tx=0x7f3a4400bc80 comp rx=0 tx=0).stop 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.161+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 shutdown_connections 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.161+0000 7f3a51f18700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a4c104040 0x7f3a4c1a2e70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.162+0000 7f3a51f18700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a380b2880 0x7f3a380b4d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.162+0000 7f3a51f18700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a4c1024a0 0x7f3a4c06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.162+0000 7f3a51f18700 1 --2- 172.21.15.77:0/3938322360 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a4c1036a0 0x7f3a4c1a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:39.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.162+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 >> 172.21.15.77:0/3938322360 conn(0x7f3a4c0fda40 msgr2=0x7f3a4c106b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.162+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 shutdown_connections 2024-05-27T08:37:39.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:39.162+0000 7f3a51f18700 1 -- 172.21.15.77:0/3938322360 wait complete. 2024-05-27T08:37:39.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:38 smithi092 ceph-mon[36049]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:39.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:38 smithi179 ceph-mon[36481]: pgmap v365: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:39.712 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-27T08:37:39.721 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:37:39.721 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-27T08:37:39.961 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:41.203 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:40 smithi077 ceph-mon[32007]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:41.203 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:40 smithi077 ceph-mon[32007]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:41.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:40 smithi092 ceph-mon[36049]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:41.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:40 smithi092 ceph-mon[36049]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:41.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:40 smithi179 ceph-mon[36481]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:37:41.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:40 smithi179 ceph-mon[36481]: pgmap v366: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:37:41.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 -- 172.21.15.77:0/4048376862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b641052a0 msgr2=0x7f2b6410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:41.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/4048376862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b641052a0 0x7f2b6410bb50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00b0a0 tx=0x7f2b5c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 -- 172.21.15.77:0/4048376862 shutdown_connections 2024-05-27T08:37:41.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/4048376862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b641052a0 0x7f2b6410bb50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/4048376862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b64101920 0x7f2b64101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/4048376862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b64104900 0x7f2b64104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 -- 172.21.15.77:0/4048376862 >> 172.21.15.77:0/4048376862 conn(0x7f2b640fd280 msgr2=0x7f2b640ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 -- 172.21.15.77:0/4048376862 shutdown_connections 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.593+0000 7f2b6acf5700 1 -- 172.21.15.77:0/4048376862 wait complete. 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b6acf5700 1 Processor -- start 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b6acf5700 1 -- start start 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b6acf5700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b64101920 0x7f2b64199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b6acf5700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b64104900 0x7f2b64199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b6acf5700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b641052a0 0x7f2b641a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b6acf5700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410f1e0 con 0x7f2b64104900 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b6acf5700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410f060 con 0x7f2b641052a0 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b6acf5700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6410f360 con 0x7f2b64101920 2024-05-27T08:37:41.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b64104900 0x7f2b64199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b64104900 0x7f2b64199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60432/0 (socket says 172.21.15.77:60432) 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 -- 172.21.15.77:0/2518536446 learned_addr learned my addr 172.21.15.77:0/2518536446 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b68a91700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b64101920 0x7f2b64199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b69292700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b641052a0 0x7f2b641a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 -- 172.21.15.77:0/2518536446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b64101920 msgr2=0x7f2b64199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b64101920 0x7f2b64199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 -- 172.21.15.77:0/2518536446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b641052a0 msgr2=0x7f2b641a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b641052a0 0x7f2b641a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 -- 172.21.15.77:0/2518536446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b641a1480 con 0x7f2b64104900 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b69292700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b641052a0 0x7f2b641a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:41.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b68a91700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b64101920 0x7f2b64199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.594+0000 7f2b63fff700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b64104900 0x7f2b64199d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000b560 tx=0x7f2b5000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.595+0000 7f2b61ffb700 1 -- 172.21.15.77:0/2518536446 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50015020 con 0x7f2b64104900 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.595+0000 7f2b61ffb700 1 -- 172.21.15.77:0/2518536446 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b50011ce0 con 0x7f2b64104900 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.595+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b641a1770 con 0x7f2b64104900 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.595+0000 7f2b61ffb700 1 -- 172.21.15.77:0/2518536446 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50010da0 con 0x7f2b64104900 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.595+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641a1ce0 con 0x7f2b64104900 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.596+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6406a230 con 0x7f2b64104900 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.597+0000 7f2b61ffb700 1 -- 172.21.15.77:0/2518536446 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2b50018020 con 0x7f2b64104900 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.598+0000 7f2b61ffb700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2b540b2880 0x7f2b540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.598+0000 7f2b68a91700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2b540b2880 0x7f2b540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.598+0000 7f2b61ffb700 1 -- 172.21.15.77:0/2518536446 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f2b500e1a40 con 0x7f2b64104900 2024-05-27T08:37:41.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.598+0000 7f2b68a91700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2b540b2880 0x7f2b540b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b6406bad0 tx=0x7f2b58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:41.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.602+0000 7f2b61ffb700 1 -- 172.21.15.77:0/2518536446 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2b500e3e20 con 0x7f2b64104900 2024-05-27T08:37:41.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.849+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2b64064150 con 0x7f2b64104900 2024-05-27T08:37:41.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.849+0000 7f2b61ffb700 1 -- 172.21.15.77:0/2518536446 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3088 (secure 0 0 0) 0x7f2b5001f020 con 0x7f2b64104900 2024-05-27T08:37:41.851 INFO:teuthology.orchestra.run.smithi077.stdout:e28 2024-05-27T08:37:41.851 INFO:teuthology.orchestra.run.smithi077.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-27T08:37:41.851 INFO:teuthology.orchestra.run.smithi077.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T08:37:41.851 INFO:teuthology.orchestra.run.smithi077.stdout:legacy client fscid: 1 2024-05-27T08:37:41.851 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:41.851 INFO:teuthology.orchestra.run.smithi077.stdout:Filesystem 'cephfs' (1) 2024-05-27T08:37:41.851 INFO:teuthology.orchestra.run.smithi077.stdout:fs_name cephfs 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 28 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-05-27T08:35:08.767215+0000 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-05-27T08:35:26.135167+0000 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:tableserver 0 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:root 0 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:session_timeout 300 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:session_autoclose 300 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:max_file_size 1099511627776 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:max_xattr_size 65536 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:required_client_features {} 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure 0 2024-05-27T08:37:41.852 INFO:teuthology.orchestra.run.smithi077.stdout:last_failure_osd_epoch 0 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:max_mds 5 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:in 0,1,2,3,4 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:up {0=24434,1=24433,2=24418,3=24412,4=24410} 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:failed 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:damaged 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:stopped 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:data_pools [3] 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:metadata_pool 2 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:inline_data disabled 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:balancer 2024-05-27T08:37:41.853 INFO:teuthology.orchestra.run.smithi077.stdout:bal_rank_mask -1 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:standby_count_wanted 0 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:qdb_cluster leader: 24434 members: 24433,24412,24410,24418,24434 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.b{0:24434} state up:active seq 5 addr [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.e{0:24389} state up:standby-replay seq 1 addr [v2:172.21.15.92:6834/4019780450,v1:172.21.15.92:6835/4019780450] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.i{1:24433} state up:active seq 11 addr [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.l{1:24385} state up:standby-replay seq 1 addr [v2:172.21.15.179:6832/1008462834,v1:172.21.15.179:6833/1008462834] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.c{2:24418} state up:active seq 12 addr [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.k{2:24365} state up:standby-replay seq 1 addr [v2:172.21.15.92:6832/1631050328,v1:172.21.15.92:6833/1631050328] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.f{3:24412} state up:active seq 13 addr [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.a{3:14595} state up:standby-replay seq 1 addr [v2:172.21.15.77:6840/906525687,v1:172.21.15.77:6841/906525687] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.h{4:24410} state up:active seq 12 addr [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.g{4:14583} state up:standby-replay seq 1 addr [v2:172.21.15.77:6838/3869213740,v1:172.21.15.77:6839/3869213740] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.854 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:41.855 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:41.855 INFO:teuthology.orchestra.run.smithi077.stdout:Standby daemons: 2024-05-27T08:37:41.855 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:41.855 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.j{-1:14520} state up:standby seq 1 addr [v2:172.21.15.77:6834/3520949826,v1:172.21.15.77:6835/3520949826] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.855 INFO:teuthology.orchestra.run.smithi077.stdout:[mds.d{-1:14532} state up:standby seq 1 addr [v2:172.21.15.77:6836/619760507,v1:172.21.15.77:6837/619760507] compat {c=[1],r=[1],i=[1fff]}] 2024-05-27T08:37:41.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.852+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2b540b2880 msgr2=0x7f2b540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.852+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2b540b2880 0x7f2b540b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2b6406bad0 tx=0x7f2b58008040 comp rx=0 tx=0).stop 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b64104900 msgr2=0x7f2b64199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b64104900 0x7f2b64199d60 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000b560 tx=0x7f2b5000b920 comp rx=0 tx=0).stop 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 shutdown_connections 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2b641052a0 0x7f2b641a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f2b540b2880 0x7f2b540b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b64101920 0x7f2b64199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 --2- 172.21.15.77:0/2518536446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b64104900 0x7f2b64199d60 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 >> 172.21.15.77:0/2518536446 conn(0x7f2b640fd280 msgr2=0x7f2b64107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 shutdown_connections 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:41.853+0000 7f2b6acf5700 1 -- 172.21.15.77:0/2518536446 wait complete. 2024-05-27T08:37:41.856 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-05-27T08:37:42.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:41 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2518536446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T08:37:42.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:41 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2518536446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T08:37:42.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:41 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2518536446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-27T08:37:42.483 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-27T08:37:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:43.115 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:42 smithi077 ceph-mon[32007]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-27T08:37:43.238 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:42 smithi092 ceph-mon[36049]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-27T08:37:43.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:42 smithi179 ceph-mon[36481]: pgmap v367: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-27T08:37:44.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.432+0000 7f3201590700 1 -- 172.21.15.77:0/2033809279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1035b0 msgr2=0x7f31fc1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:44.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.432+0000 7f3201590700 1 --2- 172.21.15.77:0/2033809279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1035b0 0x7f31fc1039d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0099c0 tx=0x7f31ec009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.432+0000 7f3201590700 1 -- 172.21.15.77:0/2033809279 shutdown_connections 2024-05-27T08:37:44.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.432+0000 7f3201590700 1 --2- 172.21.15.77:0/2033809279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31fc105150 0x7f31fc10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.432+0000 7f3201590700 1 --2- 172.21.15.77:0/2033809279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1035b0 0x7f31fc1039d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.432+0000 7f3201590700 1 --2- 172.21.15.77:0/2033809279 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc1047b0 0x7f31fc104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.432+0000 7f3201590700 1 -- 172.21.15.77:0/2033809279 >> 172.21.15.77:0/2033809279 conn(0x7f31fc0feb50 msgr2=0x7f31fc100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:44.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.433+0000 7f3201590700 1 -- 172.21.15.77:0/2033809279 shutdown_connections 2024-05-27T08:37:44.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.433+0000 7f3201590700 1 -- 172.21.15.77:0/2033809279 wait complete. 2024-05-27T08:37:44.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.433+0000 7f3201590700 1 Processor -- start 2024-05-27T08:37:44.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.433+0000 7f3201590700 1 -- start start 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f3201590700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31fc1035b0 0x7f31fc199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f3201590700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1047b0 0x7f31fc199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f3201590700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc105150 0x7f31fc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f3201590700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc10f0b0 con 0x7f31fc1047b0 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f3201590700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc10ef30 con 0x7f31fc105150 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f3201590700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc10f230 con 0x7f31fc1035b0 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1047b0 0x7f31fc199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1047b0 0x7f31fc199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60446/0 (socket says 172.21.15.77:60446) 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 -- 172.21.15.77:0/2642230993 learned_addr learned my addr 172.21.15.77:0/2642230993 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31faffd700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31fc1035b0 0x7f31fc199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fb7fe700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc105150 0x7f31fc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:44.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 -- 172.21.15.77:0/2642230993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31fc1035b0 msgr2=0x7f31fc199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31fc1035b0 0x7f31fc199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 -- 172.21.15.77:0/2642230993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc105150 msgr2=0x7f31fc1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc105150 0x7f31fc1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 -- 172.21.15.77:0/2642230993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc1a13b0 con 0x7f31fc1047b0 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31fa7fc700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1047b0 0x7f31fc199c90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f31f000d9b0 tx=0x7f31f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31e3fff700 1 -- 172.21.15.77:0/2642230993 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0016070 con 0x7f31fc1047b0 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31e3fff700 1 -- 172.21.15.77:0/2642230993 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f0004830 con 0x7f31fc1047b0 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc1a16a0 con 0x7f31fc1047b0 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.434+0000 7f31e3fff700 1 -- 172.21.15.77:0/2642230993 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0005500 con 0x7f31fc1047b0 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.435+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc1a1c30 con 0x7f31fc1047b0 2024-05-27T08:37:44.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.436+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31fc06a230 con 0x7f31fc1047b0 2024-05-27T08:37:44.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.437+0000 7f31e3fff700 1 -- 172.21.15.77:0/2642230993 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f31f0004220 con 0x7f31fc1047b0 2024-05-27T08:37:44.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.438+0000 7f31e3fff700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f31e40b2900 0x7f31e40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:44.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.438+0000 7f31faffd700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f31e40b2900 0x7f31e40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:44.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.438+0000 7f31e3fff700 1 -- 172.21.15.77:0/2642230993 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f31f00e2570 con 0x7f31fc1047b0 2024-05-27T08:37:44.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.438+0000 7f31faffd700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f31e40b2900 0x7f31e40b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31ec006010 tx=0x7f31ec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:44.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.441+0000 7f31e3fff700 1 -- 172.21.15.77:0/2642230993 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31f001f020 con 0x7f31fc1047b0 2024-05-27T08:37:44.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.647+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f31fc02e590 con 0x7f31fc1047b0 2024-05-27T08:37:44.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.648+0000 7f31e3fff700 1 -- 172.21.15.77:0/2642230993 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v85) ==== 56+0+5302 (secure 0 0 0) 0x7f31f00aa290 con 0x7f31fc1047b0 2024-05-27T08:37:44.649 INFO:teuthology.orchestra.run.smithi077.stdout:epoch 85 2024-05-27T08:37:44.649 INFO:teuthology.orchestra.run.smithi077.stdout:fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:37:44.649 INFO:teuthology.orchestra.run.smithi077.stdout:created 2024-05-27T08:26:55.341917+0000 2024-05-27T08:37:44.649 INFO:teuthology.orchestra.run.smithi077.stdout:modified 2024-05-27T08:35:58.361436+0000 2024-05-27T08:37:44.649 INFO:teuthology.orchestra.run.smithi077.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:crush_version 26 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:full_ratio 0.95 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:backfillfull_ratio 0.9 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:nearfull_ratio 0.85 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:require_min_compat_client luminous 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:min_compat_client luminous 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:require_osd_release squid 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:stretch_mode_enabled false 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-27T08:37:44.650 INFO:teuthology.orchestra.run.smithi077.stdout:max_osd 12 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.0 up in weight 1 up_from 8 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6802/3594583548,v1:172.21.15.77:6803/3594583548] [v2:172.21.15.77:6804/3594583548,v1:172.21.15.77:6805/3594583548] exists,up 2957ea67-5b11-4557-ab6d-ec841890c830 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6800/4268921047,v1:172.21.15.92:6801/4268921047] [v2:172.21.15.92:6802/4268921047,v1:172.21.15.92:6803/4268921047] exists,up 128362ac-83fc-46b5-a7aa-a7c48e49c0d2 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6800/2614728525,v1:172.21.15.179:6801/2614728525] [v2:172.21.15.179:6802/2614728525,v1:172.21.15.179:6803/2614728525] exists,up 8ae8a416-499c-4a05-9389-9dac4e5415de 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.3 up in weight 1 up_from 27 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6810/2112674768,v1:172.21.15.77:6811/2112674768] [v2:172.21.15.77:6812/2112674768,v1:172.21.15.77:6813/2112674768] exists,up 96e70a03-bba7-4d18-a3f4-d14c454f719f 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.4 up in weight 1 up_from 33 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6808/2588881840,v1:172.21.15.92:6809/2588881840] [v2:172.21.15.92:6810/2588881840,v1:172.21.15.92:6811/2588881840] exists,up d93ff380-9258-4203-9826-6ddedfa1d03b 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.5 up in weight 1 up_from 40 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6808/1761235705,v1:172.21.15.179:6809/1761235705] [v2:172.21.15.179:6810/1761235705,v1:172.21.15.179:6811/1761235705] exists,up c05fe340-efa9-43dd-ac86-7d0f436df4a2 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.6 up in weight 1 up_from 46 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6818/3753560520,v1:172.21.15.77:6819/3753560520] [v2:172.21.15.77:6820/3753560520,v1:172.21.15.77:6821/3753560520] exists,up f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.7 up in weight 1 up_from 51 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6816/3593066168,v1:172.21.15.92:6817/3593066168] [v2:172.21.15.92:6818/3593066168,v1:172.21.15.92:6819/3593066168] exists,up f6262ef1-767f-4b5f-a376-f6a4cb631146 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.8 up in weight 1 up_from 57 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6816/3383738414,v1:172.21.15.179:6817/3383738414] [v2:172.21.15.179:6818/3383738414,v1:172.21.15.179:6819/3383738414] exists,up 018d4143-7dfd-40d3-9b26-758ce6acb695 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.9 up in weight 1 up_from 63 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6826/4049231537,v1:172.21.15.77:6827/4049231537] [v2:172.21.15.77:6828/4049231537,v1:172.21.15.77:6829/4049231537] exists,up 058309ee-c447-48af-aec5-768f8c9f5222 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.10 up in weight 1 up_from 69 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.92:6824/2191388192,v1:172.21.15.92:6825/2191388192] [v2:172.21.15.92:6826/2191388192,v1:172.21.15.92:6827/2191388192] exists,up 0a68bc35-c24b-4bf2-911f-2f711c0bd59a 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:osd.11 up in weight 1 up_from 75 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6824/113533236,v1:172.21.15.179:6825/113533236] [v2:172.21.15.179:6826/113533236,v1:172.21.15.179:6827/113533236] exists,up 1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b 2024-05-27T08:37:44.651 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.4 [2,11] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.10 [2,4] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.1c [8,11] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 2.1f [8,1] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.e [4,8] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.1b [5,9] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.1e [3,9] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.24 [4,9] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:pg_upmap_items 3.27 [4,6] 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/2684021753 expires 2024-05-28T08:27:57.064761+0000 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6800/2456305222 expires 2024-05-28T08:27:57.064761+0000 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/3434063639 expires 2024-05-28T08:27:57.064761+0000 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6801/2456305222 expires 2024-05-28T08:27:57.064761+0000 2024-05-27T08:37:44.652 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6801/2203620437 expires 2024-05-28T08:27:30.012493+0000 2024-05-27T08:37:44.653 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/21678968 expires 2024-05-28T08:27:30.012493+0000 2024-05-27T08:37:44.653 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/3150137608 expires 2024-05-28T08:27:57.064761+0000 2024-05-27T08:37:44.653 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:6800/2203620437 expires 2024-05-28T08:27:30.012493+0000 2024-05-27T08:37:44.653 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/1391732673 expires 2024-05-28T08:27:30.012493+0000 2024-05-27T08:37:44.653 INFO:teuthology.orchestra.run.smithi077.stdout:blocklist 172.21.15.77:0/3386760140 expires 2024-05-28T08:27:30.012493+0000 2024-05-27T08:37:44.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f31e40b2900 msgr2=0x7f31e40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:44.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f31e40b2900 0x7f31e40b4dc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f31ec006010 tx=0x7f31ec000bc0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1047b0 msgr2=0x7f31fc199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:44.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1047b0 0x7f31fc199c90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f31f000d9b0 tx=0x7f31f000dcc0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 shutdown_connections 2024-05-27T08:37:44.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f31fc105150 0x7f31fc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f31e40b2900 0x7f31e40b4dc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31fc1035b0 0x7f31fc199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 --2- 172.21.15.77:0/2642230993 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f31fc1047b0 0x7f31fc199c90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:44.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 >> 172.21.15.77:0/2642230993 conn(0x7f31fc0feb50 msgr2=0x7f31fc107b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:44.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 shutdown_connections 2024-05-27T08:37:44.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:44.651+0000 7f3201590700 1 -- 172.21.15.77:0/2642230993 wait complete. 2024-05-27T08:37:45.218 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:44 smithi077 ceph-mon[32007]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T08:37:45.219 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:44 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2642230993' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T08:37:45.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:44 smithi092 ceph-mon[36049]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T08:37:45.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:44 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2642230993' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T08:37:45.246 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-27T08:37:45.258 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:37:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:44 smithi179 ceph-mon[36481]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-05-27T08:37:45.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:44 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2642230993' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-27T08:37:45.500 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:47.146 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:46 smithi077 ceph-mon[32007]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-27T08:37:47.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 -- 172.21.15.77:0/1256860692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102df0 msgr2=0x7f739410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:47.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 --2- 172.21.15.77:0/1256860692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102df0 0x7f739410f2e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f738c00b0a0 tx=0x7f738c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 -- 172.21.15.77:0/1256860692 shutdown_connections 2024-05-27T08:37:47.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 --2- 172.21.15.77:0/1256860692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102df0 0x7f739410f2e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 --2- 172.21.15.77:0/1256860692 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7394108510 0x7f73941088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 --2- 172.21.15.77:0/1256860692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7394102430 0x7f73941028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 -- 172.21.15.77:0/1256860692 >> 172.21.15.77:0/1256860692 conn(0x7f73940793b0 msgr2=0x7f73940797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:47.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 -- 172.21.15.77:0/1256860692 shutdown_connections 2024-05-27T08:37:47.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.151+0000 7f739b297700 1 -- 172.21.15.77:0/1256860692 wait complete. 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f739b297700 1 Processor -- start 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f739b297700 1 -- start start 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f739b297700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102430 0x7f739419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f739b297700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7394102df0 0x7f739419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f739b297700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7394108510 0x7f73941999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f739b297700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f73941125e0 con 0x7f7394102430 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f739b297700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7394112460 con 0x7f7394102df0 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f739b297700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7394112760 con 0x7f7394108510 2024-05-27T08:37:47.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102430 0x7f739419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102430 0x7f739419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60468/0 (socket says 172.21.15.77:60468) 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 -- 172.21.15.77:0/3434471651 learned_addr learned my addr 172.21.15.77:0/3434471651 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 -- 172.21.15.77:0/3434471651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7394108510 msgr2=0x7f73941999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7394108510 0x7f73941999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 -- 172.21.15.77:0/3434471651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7394102df0 msgr2=0x7f739419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7394102df0 0x7f739419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 -- 172.21.15.77:0/3434471651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739419a120 con 0x7f7394102430 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.152+0000 7f7399033700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102430 0x7f739419f800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f739000d950 tx=0x7f739000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.153+0000 7f738a7fc700 1 -- 172.21.15.77:0/3434471651 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390016070 con 0x7f7394102430 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.153+0000 7f738a7fc700 1 -- 172.21.15.77:0/3434471651 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7390010460 con 0x7f7394102430 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.153+0000 7f738a7fc700 1 -- 172.21.15.77:0/3434471651 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739000f620 con 0x7f7394102430 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.153+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739419a410 con 0x7f7394102430 2024-05-27T08:37:47.156 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.153+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941a66d0 con 0x7f7394102430 2024-05-27T08:37:47.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.154+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f739406a230 con 0x7f7394102430 2024-05-27T08:37:47.157 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.155+0000 7f738a7fc700 1 -- 172.21.15.77:0/3434471651 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7390019070 con 0x7f7394102430 2024-05-27T08:37:47.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.156+0000 7f738a7fc700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f73800b2900 0x7f73800b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:47.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.156+0000 7f7398832700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f73800b2900 0x7f73800b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:47.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.156+0000 7f738a7fc700 1 -- 172.21.15.77:0/3434471651 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f73900e1db0 con 0x7f7394102430 2024-05-27T08:37:47.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.156+0000 7f7398832700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f73800b2900 0x7f73800b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7384005950 tx=0x7f73840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:47.161 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.160+0000 7f738a7fc700 1 -- 172.21.15.77:0/3434471651 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73900a9a90 con 0x7f7394102430 2024-05-27T08:37:47.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:46 smithi092 ceph-mon[36049]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-27T08:37:47.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:46 smithi179 ceph-mon[36481]: pgmap v369: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-05-27T08:37:47.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.418+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f739402e560 con 0x7f7394102430 2024-05-27T08:37:47.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.419+0000 7f738a7fc700 1 -- 172.21.15.77:0/3434471651 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11927 (secure 0 0 0) 0x7f73900a9390 con 0x7f7394102430 2024-05-27T08:37:47.421 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:47.421 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:35:26.135167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24433,24412,24410,24418,24434]},"id":1}]} 2024-05-27T08:37:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f73800b2900 msgr2=0x7f73800b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f73800b2900 0x7f73800b4dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7384005950 tx=0x7f73840058e0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102430 msgr2=0x7f739419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102430 0x7f739419f800 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f739000d950 tx=0x7f739000dd10 comp rx=0 tx=0).stop 2024-05-27T08:37:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 shutdown_connections 2024-05-27T08:37:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f73800b2900 0x7f73800b4dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7394108510 0x7f73941999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7394102430 0x7f739419f800 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 --2- 172.21.15.77:0/3434471651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7394102df0 0x7f739419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.421+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 >> 172.21.15.77:0/3434471651 conn(0x7f73940793b0 msgr2=0x7f739406bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:47.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.422+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 shutdown_connections 2024-05-27T08:37:47.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:47.422+0000 7f739b297700 1 -- 172.21.15.77:0/3434471651 wait complete. 2024-05-27T08:37:47.424 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-05-27T08:37:48.001 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:47 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3434471651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:48.030 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:37:48.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:47 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3434471651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:48.277 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:48.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:47 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3434471651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:49.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:48 smithi077 ceph-mon[32007]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-27T08:37:49.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:48 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:37:49.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:48 smithi092 ceph-mon[36049]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-27T08:37:49.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:48 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:37:49.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:48 smithi179 ceph-mon[36481]: pgmap v370: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 8 op/s 2024-05-27T08:37:49.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:48 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:37:50.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 -- 172.21.15.77:0/652331798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618108520 msgr2=0x7f7618108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:50.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 --2- 172.21.15.77:0/652331798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618108520 0x7f7618108900 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7614009990 tx=0x7f7614009ca0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 -- 172.21.15.77:0/652331798 shutdown_connections 2024-05-27T08:37:50.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 --2- 172.21.15.77:0/652331798 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7618102e00 0x7f761810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 --2- 172.21.15.77:0/652331798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618108520 0x7f7618108900 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 --2- 172.21.15.77:0/652331798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618102440 0x7f76181028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 -- 172.21.15.77:0/652331798 >> 172.21.15.77:0/652331798 conn(0x7f76180793f0 msgr2=0x7f7618079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:50.002 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 -- 172.21.15.77:0/652331798 shutdown_connections 2024-05-27T08:37:50.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.000+0000 7f761ff81700 1 -- 172.21.15.77:0/652331798 wait complete. 2024-05-27T08:37:50.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.001+0000 7f761ff81700 1 Processor -- start 2024-05-27T08:37:50.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.001+0000 7f761ff81700 1 -- start start 2024-05-27T08:37:50.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.001+0000 7f761ff81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618102440 0x7f761819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:50.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.001+0000 7f761ff81700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7618102e00 0x7f761819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:50.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.001+0000 7f761ff81700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618108520 0x7f76181989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:50.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.001+0000 7f761ff81700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76181125f0 con 0x7f7618102440 2024-05-27T08:37:50.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.001+0000 7f761ff81700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7618112470 con 0x7f7618102e00 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.001+0000 7f761ff81700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7618112770 con 0x7f7618108520 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618102440 0x7f761819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618102440 0x7f761819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59552/0 (socket says 172.21.15.77:59552) 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 -- 172.21.15.77:0/1873085492 learned_addr learned my addr 172.21.15.77:0/1873085492 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761d51c700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7618102e00 0x7f761819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761e51e700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618108520 0x7f76181989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 -- 172.21.15.77:0/1873085492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618108520 msgr2=0x7f76181989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618108520 0x7f76181989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 -- 172.21.15.77:0/1873085492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7618102e00 msgr2=0x7f761819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7618102e00 0x7f761819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 -- 172.21.15.77:0/1873085492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7618199130 con 0x7f7618102440 2024-05-27T08:37:50.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761dd1d700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618102440 0x7f761819e810 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f76140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:50.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f760effd700 1 -- 172.21.15.77:0/1873085492 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614029070 con 0x7f7618102440 2024-05-27T08:37:50.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76181993c0 con 0x7f7618102440 2024-05-27T08:37:50.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f760effd700 1 -- 172.21.15.77:0/1873085492 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7614019cc0 con 0x7f7618102440 2024-05-27T08:37:50.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f760effd700 1 -- 172.21.15.77:0/1873085492 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76140041c0 con 0x7f7618102440 2024-05-27T08:37:50.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.002+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761806b220 con 0x7f7618102440 2024-05-27T08:37:50.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.003+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761806a230 con 0x7f7618102440 2024-05-27T08:37:50.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.005+0000 7f760effd700 1 -- 172.21.15.77:0/1873085492 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f761402c070 con 0x7f7618102440 2024-05-27T08:37:50.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.006+0000 7f760effd700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:50.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.006+0000 7f761d51c700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:50.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.006+0000 7f760effd700 1 -- 172.21.15.77:0/1873085492 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f76140e4fb0 con 0x7f7618102440 2024-05-27T08:37:50.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.007+0000 7f761d51c700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7618107470 tx=0x7f760800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:50.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.011+0000 7f760effd700 1 -- 172.21.15.77:0/1873085492 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7614013070 con 0x7f7618102440 2024-05-27T08:37:50.271 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.270+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f761806bc50 con 0x7f7618102440 2024-05-27T08:37:50.272 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.271+0000 7f760effd700 1 -- 172.21.15.77:0/1873085492 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11927 (secure 0 0 0) 0x7f76140a6e20 con 0x7f7618102440 2024-05-27T08:37:50.272 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:50.272 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:35:26.135167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24433,24412,24410,24418,24434]},"id":1}]} 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 msgr2=0x7f76040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7618107470 tx=0x7f760800b3c0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618102440 msgr2=0x7f761819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618102440 0x7f761819e810 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f76140047b0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 shutdown_connections 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f76040b2880 0x7f76040b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7618108520 0x7f76181989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7618102440 0x7f761819e810 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.273+0000 7f761ff81700 1 --2- 172.21.15.77:0/1873085492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7618102e00 0x7f761819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:50.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.274+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 >> 172.21.15.77:0/1873085492 conn(0x7f76180793f0 msgr2=0x7f7618101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.274+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 shutdown_connections 2024-05-27T08:37:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:50.274+0000 7f761ff81700 1 -- 172.21.15.77:0/1873085492 wait complete. 2024-05-27T08:37:50.276 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 28 2024-05-27T08:37:50.899 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:50 smithi077 ceph-mon[32007]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T08:37:50.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:50.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:50.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:50 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1873085492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:50.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:50.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:50.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:50.900 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:50 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:50.927 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:37:51.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:50 smithi179 ceph-mon[36481]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T08:37:51.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:50 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1873085492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:51.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:50 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:50 smithi092 ceph-mon[36049]: pgmap v371: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-05-27T08:37:51.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:50 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1873085492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:37:51.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:50 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:51.851 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:53.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:52 smithi179 ceph-mon[36481]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-27T08:37:53.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:52 smithi077 ceph-mon[32007]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-27T08:37:53.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:52 smithi092 ceph-mon[36049]: pgmap v372: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-05-27T08:37:53.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 -- 172.21.15.77:0/3784468922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 msgr2=0x7fe82c108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 --2- 172.21.15.77:0/3784468922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 0x7fe82c108b10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe8280099c0 tx=0x7fe828009cd0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 -- 172.21.15.77:0/3784468922 shutdown_connections 2024-05-27T08:37:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 --2- 172.21.15.77:0/3784468922 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c1039d0 0x7fe82c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 --2- 172.21.15.77:0/3784468922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 0x7fe82c108b10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 --2- 172.21.15.77:0/3784468922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c103010 0x7fe82c103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 -- 172.21.15.77:0/3784468922 >> 172.21.15.77:0/3784468922 conn(0x7fe82c077780 msgr2=0x7fe82c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:53.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 -- 172.21.15.77:0/3784468922 shutdown_connections 2024-05-27T08:37:53.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 -- 172.21.15.77:0/3784468922 wait complete. 2024-05-27T08:37:53.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.532+0000 7fe833a50700 1 Processor -- start 2024-05-27T08:37:53.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe833a50700 1 -- start start 2024-05-27T08:37:53.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe833a50700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c103010 0x7fe82c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:53.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe833a50700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c1039d0 0x7fe82c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:53.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe833a50700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 0x7fe82c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:53.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe833a50700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c1127b0 con 0x7fe82c108730 2024-05-27T08:37:53.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe833a50700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c112630 con 0x7fe82c1039d0 2024-05-27T08:37:53.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe833a50700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c112930 con 0x7fe82c103010 2024-05-27T08:37:53.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 0x7fe82c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:53.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 0x7fe82c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59576/0 (socket says 172.21.15.77:59576) 2024-05-27T08:37:53.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 -- 172.21.15.77:0/1405832049 learned_addr learned my addr 172.21.15.77:0/1405832049 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:53.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe830feb700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c1039d0 0x7fe82c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:53.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe8317ec700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c103010 0x7fe82c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:53.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 -- 172.21.15.77:0/1405832049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c103010 msgr2=0x7fe82c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:53.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c103010 0x7fe82c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.539 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 -- 172.21.15.77:0/1405832049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c1039d0 msgr2=0x7fe82c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:53.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c1039d0 0x7fe82c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 -- 172.21.15.77:0/1405832049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c1991d0 con 0x7fe82c108730 2024-05-27T08:37:53.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.533+0000 7fe831fed700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 0x7fe82c198a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe8240094f0 tx=0x7fe82400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:53.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.534+0000 7fe8227fc700 1 -- 172.21.15.77:0/1405832049 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82400ddd0 con 0x7fe82c108730 2024-05-27T08:37:53.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.534+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82c1994c0 con 0x7fe82c108730 2024-05-27T08:37:53.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.534+0000 7fe8227fc700 1 -- 172.21.15.77:0/1405832049 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe824009e20 con 0x7fe82c108730 2024-05-27T08:37:53.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.534+0000 7fe8227fc700 1 -- 172.21.15.77:0/1405832049 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824015400 con 0x7fe82c108730 2024-05-27T08:37:53.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.534+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82c1a5780 con 0x7fe82c108730 2024-05-27T08:37:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.535+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82c10aaa0 con 0x7fe82c108730 2024-05-27T08:37:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.536+0000 7fe8227fc700 1 -- 172.21.15.77:0/1405832049 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe824004dd0 con 0x7fe82c108730 2024-05-27T08:37:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.537+0000 7fe8227fc700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe8180b2880 0x7fe8180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.537+0000 7fe8317ec700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe8180b2880 0x7fe8180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.537+0000 7fe8227fc700 1 -- 172.21.15.77:0/1405832049 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe8240e2460 con 0x7fe82c108730 2024-05-27T08:37:53.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.537+0000 7fe8317ec700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe8180b2880 0x7fe8180b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe828000c00 tx=0x7fe828005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:53.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.540+0000 7fe8227fc700 1 -- 172.21.15.77:0/1405832049 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe824010070 con 0x7fe82c108730 2024-05-27T08:37:53.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.767+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe82c199fa0 con 0x7fe82c108730 2024-05-27T08:37:53.770 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.769+0000 7fe8227fc700 1 -- 172.21.15.77:0/1405832049 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22676 (secure 0 0 0) 0x7fe8240b2030 con 0x7fe82c108730 2024-05-27T08:37:53.771 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:37:53.771 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":85,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:35:58.361436+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe8180b2880 msgr2=0x7fe8180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe8180b2880 0x7fe8180b4d40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe828000c00 tx=0x7fe828005fb0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 msgr2=0x7fe82c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 0x7fe82c198a70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe8240094f0 tx=0x7fe82400d3b0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 shutdown_connections 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe82c108730 0x7fe82c198a70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe8180b2880 0x7fe8180b4d40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe82c103010 0x7fe82c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 --2- 172.21.15.77:0/1405832049 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe82c1039d0 0x7fe82c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 >> 172.21.15.77:0/1405832049 conn(0x7fe82c077780 msgr2=0x7fe82c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:53.774 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 shutdown_connections 2024-05-27T08:37:53.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:53.772+0000 7fe833a50700 1 -- 172.21.15.77:0/1405832049 wait complete. 2024-05-27T08:37:54.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T08:35:08.767215+0000', 'modified': '2024-05-27T08:35:26.135167+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24434, 'mds_1': 24433, 'mds_2': 24418, 'mds_3': 24412, 'mds_4': 24410}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/3869213740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 3869213740}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 3869213740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6841/906525687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 906525687}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 906525687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24365': {'gid': 24365, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6833/1631050328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 1631050328}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 1631050328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6833/1008462834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 1008462834}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 1008462834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/4019780450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 4019780450}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 4019780450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/2920087591', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 2920087591}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2920087591}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.179:6835/2178851321', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 2178851321}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 2178851321}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.179:6837/2857088493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 2857088493}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 2857088493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.179:6839/2672999131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2672999131}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2672999131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/709031605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 709031605}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 709031605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24434, 'qdb_cluster': [24433, 24412, 24410, 24418, 24434]} 2024-05-27T08:37:54.496 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-27T08:37:54.496 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T08:37:54.754 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:55.080 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:54 smithi077 ceph-mon[32007]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:55.080 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:55.080 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:55.080 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:54 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1405832049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:37:55.080 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:37:55.080 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:37:55.080 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:54 smithi077 ceph-mon[32007]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:55.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:54 smithi179 ceph-mon[36481]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:55.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:54 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1405832049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:37:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:37:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:37:55.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:54 smithi179 ceph-mon[36481]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:55.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:54 smithi092 ceph-mon[36049]: pgmap v373: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:37:55.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:55.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:55.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:54 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1405832049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:37:55.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:37:55.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:37:55.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:54 smithi092 ceph-mon[36049]: from='mgr.14152 172.21.15.77:0/1435865952' entity='mgr.x' 2024-05-27T08:37:56.474 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 -- 172.21.15.77:0/2285570907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806c600 msgr2=0x7fd01810fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2285570907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806c600 0x7fd01810fab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009990 tx=0x7fd00c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 -- 172.21.15.77:0/2285570907 shutdown_connections 2024-05-27T08:37:56.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2285570907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd01810fff0 0x7fd0181123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2285570907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01806bce0 0x7fd01806c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2285570907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806c600 0x7fd01810fab0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 -- 172.21.15.77:0/2285570907 >> 172.21.15.77:0/2285570907 conn(0x7fd0180793f0 msgr2=0x7fd018079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:56.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 -- 172.21.15.77:0/2285570907 shutdown_connections 2024-05-27T08:37:56.475 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.473+0000 7fd01cf38700 1 -- 172.21.15.77:0/2285570907 wait complete. 2024-05-27T08:37:56.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01cf38700 1 Processor -- start 2024-05-27T08:37:56.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01cf38700 1 -- start start 2024-05-27T08:37:56.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01cf38700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806bce0 0x7fd01819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01cf38700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd01806c600 0x7fd01819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01cf38700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01810fff0 0x7fd0181a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01cf38700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd018115930 con 0x7fd01806bce0 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01cf38700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181157b0 con 0x7fd01810fff0 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01cf38700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd018115ab0 con 0x7fd01806c600 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806bce0 0x7fd01819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806bce0 0x7fd01819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59600/0 (socket says 172.21.15.77:59600) 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01659c700 1 -- 172.21.15.77:0/2302357343 learned_addr learned my addr 172.21.15.77:0/2302357343 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd015d9b700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd01806c600 0x7fd01819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd016d9d700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01810fff0 0x7fd0181a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.474+0000 7fd01659c700 1 -- 172.21.15.77:0/2302357343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd01806c600 msgr2=0x7fd01819c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd01659c700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd01806c600 0x7fd01819c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd01659c700 1 -- 172.21.15.77:0/2302357343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01810fff0 msgr2=0x7fd0181a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.478 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd01659c700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01810fff0 0x7fd0181a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd01659c700 1 -- 172.21.15.77:0/2302357343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0181a0e20 con 0x7fd01806bce0 2024-05-27T08:37:56.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd01659c700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806bce0 0x7fd01819be60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd00800d950 tx=0x7fd00800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:56.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd0077fe700 1 -- 172.21.15.77:0/2302357343 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008016070 con 0x7fd01806bce0 2024-05-27T08:37:56.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd01cf38700 1 -- 172.21.15.77:0/2302357343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0181a1110 con 0x7fd01806bce0 2024-05-27T08:37:56.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd0077fe700 1 -- 172.21.15.77:0/2302357343 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00800fd40 con 0x7fd01806bce0 2024-05-27T08:37:56.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd0077fe700 1 -- 172.21.15.77:0/2302357343 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00800e3b0 con 0x7fd01806bce0 2024-05-27T08:37:56.479 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.475+0000 7fd01cf38700 1 -- 172.21.15.77:0/2302357343 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0181a8990 con 0x7fd01806bce0 2024-05-27T08:37:56.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.478+0000 7fd0077fe700 1 -- 172.21.15.77:0/2302357343 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd008019070 con 0x7fd01806bce0 2024-05-27T08:37:56.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd0077fe700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd0000b2900 0x7fd0000b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd015d9b700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd0000b2900 0x7fd0000b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd0077fe700 1 -- 172.21.15.77:0/2302357343 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd0080e1f90 con 0x7fd01806bce0 2024-05-27T08:37:56.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd0277ceb80 1 Processor -- start 2024-05-27T08:37:56.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd015d9b700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd0000b2900 0x7fd0000b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00b5c0 tx=0x7fd00c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd0277ceb80 1 -- start start 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e30f810 0x55f11e30fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 0x55f11e3138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.479+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e313f10 0x55f11e314350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f11e26ec40 con 0x55f11e313f10 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f11e25e7e0 con 0x55f11e30f810 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f11e262820 con 0x55f11e3101c0 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e313f10 0x55f11e314350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e313f10 0x55f11e314350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59602/0 (socket says 172.21.15.77:59602) 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 -- 172.21.15.77:0/1304654500 learned_addr learned my addr 172.21.15.77:0/1304654500 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd01659c700 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e30f810 0x55f11e30fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.485 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd015d9b700 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 0x55f11e3138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 -- 172.21.15.77:0/1304654500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 msgr2=0x55f11e3138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 0x55f11e3138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 -- 172.21.15.77:0/1304654500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e30f810 msgr2=0x55f11e30fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e30f810 0x55f11e30fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 -- 172.21.15.77:0/1304654500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11e1ae880 con 0x55f11e313f10 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd01659c700 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e30f810 0x55f11e30fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd016d9d700 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e313f10 0x55f11e314350 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd010003f40 tx=0x7fd010009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ed8c909b9af793 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd004ff9700 1 -- 172.21.15.77:0/1304654500 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010004bb0 con 0x55f11e313f10 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd004ff9700 1 -- 172.21.15.77:0/1304654500 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd010004d50 con 0x55f11e313f10 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.480+0000 7fd004ff9700 1 -- 172.21.15.77:0/1304654500 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010007e00 con 0x55f11e313f10 2024-05-27T08:37:56.486 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1304654500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e313f10 msgr2=0x55f11e314350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e313f10 0x55f11e314350 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd010003f40 tx=0x7fd010009ba0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1304654500 shutdown_connections 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e313f10 0x55f11e314350 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e30f810 0x55f11e30fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1304654500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 0x55f11e3138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1304654500 >> 172.21.15.77:0/1304654500 conn(0x55f11e23d030 msgr2=0x55f11e309e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1304654500 shutdown_connections 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1304654500 wait complete. 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.481+0000 7fd0277ceb80 1 -- start start 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd0277ceb80 1 Processor -- start 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e31b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 0x55f11e3185c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 0x55f11e318b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f11e164f50 con 0x55f11e30f810 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f11e200770 con 0x55f11e313f10 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f11e31c0d0 con 0x55f11e3101c0 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd0277ceb80 10 client.0 _ll_register_callbacks cb 0x55f11e1f98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e31b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e31b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59612/0 (socket says 172.21.15.77:59612) 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 -- 172.21.15.77:0/1453460859 learned_addr learned my addr 172.21.15.77:0/1453460859 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd015d9b700 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 0x55f11e3185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 -- 172.21.15.77:0/1453460859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 msgr2=0x55f11e3185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 0x55f11e3185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 -- 172.21.15.77:0/1453460859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 msgr2=0x55f11e318b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd016d9d700 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 0x55f11e318b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 0x55f11e318b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.488 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 -- 172.21.15.77:0/1453460859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11e1fd640 con 0x55f11e30f810 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fd01659c700 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e31b2c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd0080a5630 tx=0x7fd0080a9de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d04ebbae8162e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fcffdffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.482+0000 7fcffdffb700 1 -- 172.21.15.77:0/1453460859 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008020040 con 0x55f11e30f810 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fcffdffb700 1 -- 172.21.15.77:0/1453460859 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd0080106a0 con 0x55f11e30f810 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1453460859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd010004d50 con 0x55f11e30f810 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fcffdffb700 1 -- 172.21.15.77:0/1453460859 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0080b2410 con 0x55f11e30f810 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 10 client.15096 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1453460859 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f11e326780 con 0x55f11e30f810 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fcffdffb700 1 -- 172.21.15.77:0/1453460859 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd010004d50 con 0x55f11e30f810 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fcffdffb700 1 -- 172.21.15.77:0/1453460859 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd0080b0ce0 con 0x55f11e30f810 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 10 client.15096 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T08:37:56.489 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 20 client.15096 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:56.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55f11e328dd0 0x55f11e32a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 4 client.15096 mds_command: new command op to 24410 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:37:56.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.483+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1453460859 --> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f11e2db990 con 0x55f11e328dd0 2024-05-27T08:37:56.490 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.484+0000 7fd015d9b700 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55f11e328dd0 0x55f11e32a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.484+0000 7fcffdffb700 10 client.15096 ms_handle_connect on v2:172.21.15.92:6836/2920087591 2024-05-27T08:37:56.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.484+0000 7fd015d9b700 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55f11e328dd0 0x55f11e32a240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d41b9743ee64efb7 server_cookie=4c703cad877fa7a8 in_seq=0 out_seq=0 2024-05-27T08:37:56.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fcffdffb700 1 -- 172.21.15.77:0/1453460859 <== mds.4 v2:172.21.15.92:6836/2920087591 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f11e2db990 con 0x55f11e328dd0 2024-05-27T08:37:56.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fcffdffb700 10 client.15096 handle_command_reply: tid=0 2024-05-27T08:37:56.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 client.15096 shutdown 2024-05-27T08:37:56.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1453460859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 msgr2=0x55f11e31b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e31b2c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd0080a5630 tx=0x7fd0080a9de0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1453460859 shutdown_connections 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 0x55f11e318b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55f11e328dd0 0x55f11e32a240 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e31b2c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fcffdffb700 0 client.15096 ms_handle_reset on v2:172.21.15.92:6836/2920087591 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/1453460859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e3101c0 0x55f11e3185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1453460859 >> 172.21.15.77:0/1453460859 conn(0x55f11e23d030 msgr2=0x55f11e315300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1453460859 shutdown_connections 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 1 -- 172.21.15.77:0/1453460859 wait complete. 2024-05-27T08:37:56.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.485+0000 7fd0277ceb80 20 client.15096 trim_cache size 0 max 16384 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd0277ceb80 1 Processor -- start 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd0277ceb80 1 -- start start 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e30f810 0x55f11e319190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e3101c0 0x55f11e318b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 0x55f11e309c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f11e328580 con 0x55f11e3101c0 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f11e328c50 con 0x55f11e313f10 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f11e329320 con 0x55f11e30f810 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e3101c0 0x55f11e318b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e3101c0 0x55f11e318b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59614/0 (socket says 172.21.15.77:59614) 2024-05-27T08:37:56.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 -- 172.21.15.77:0/849864068 learned_addr learned my addr 172.21.15.77:0/849864068 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 -- 172.21.15.77:0/849864068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e30f810 msgr2=0x55f11e319190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 --2- 172.21.15.77:0/849864068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e30f810 0x55f11e319190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 -- 172.21.15.77:0/849864068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 msgr2=0x55f11e309c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 --2- 172.21.15.77:0/849864068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 0x55f11e309c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 -- 172.21.15.77:0/849864068 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11e30a910 con 0x55f11e3101c0 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.506+0000 7fd015d9b700 1 --2- 172.21.15.77:0/849864068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e3101c0 0x55f11e318b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009990 tx=0x7fd00c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaf1615ceea0162 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fcffffff700 1 -- 172.21.15.77:0/849864068 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c024a60 con 0x55f11e3101c0 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fcffffff700 1 -- 172.21.15.77:0/849864068 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00c00be60 con 0x55f11e3101c0 2024-05-27T08:37:56.509 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fcffffff700 1 -- 172.21.15.77:0/849864068 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c03a430 con 0x55f11e3101c0 2024-05-27T08:37:56.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 -- 172.21.15.77:0/849864068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e3101c0 msgr2=0x55f11e318b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/849864068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e3101c0 0x55f11e318b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009990 tx=0x7fd00c024000 comp rx=0 tx=0).stop 2024-05-27T08:37:56.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 -- 172.21.15.77:0/849864068 shutdown_connections 2024-05-27T08:37:56.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/849864068 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e313f10 0x55f11e309c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/849864068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e30f810 0x55f11e319190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/849864068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e3101c0 0x55f11e318b70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 -- 172.21.15.77:0/849864068 >> 172.21.15.77:0/849864068 conn(0x55f11e23d030 msgr2=0x55f11e30eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:56.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 -- 172.21.15.77:0/849864068 shutdown_connections 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 -- 172.21.15.77:0/849864068 wait complete. 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.507+0000 7fd0277ceb80 1 -- start start 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd0277ceb80 1 Processor -- start 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e333b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e3101c0 0x55f11e32e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd0277ceb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e313f10 0x55f11e32ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f11e32a0c0 con 0x55f11e30f810 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f11e3299f0 con 0x55f11e3101c0 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd0277ceb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f11e32a790 con 0x55f11e313f10 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd0277ceb80 10 client.0 _ll_register_callbacks cb 0x55f11e1f98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e333b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e333b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59616/0 (socket says 172.21.15.77:59616) 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 -- 172.21.15.77:0/836422764 learned_addr learned my addr 172.21.15.77:0/836422764 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd015d9b700 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e3101c0 0x55f11e32e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd016d9d700 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e313f10 0x55f11e32ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 -- 172.21.15.77:0/836422764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e313f10 msgr2=0x55f11e32ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e313f10 0x55f11e32ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 -- 172.21.15.77:0/836422764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e3101c0 msgr2=0x55f11e32e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e3101c0 0x55f11e32e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 -- 172.21.15.77:0/836422764 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11e32f0f0 con 0x55f11e30f810 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fd01659c700 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e333b20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0080a8f60 tx=0x7fd0080ca3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5cd6b45eb6ca39 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.508+0000 7fcffdffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fcffdffb700 1 -- 172.21.15.77:0/836422764 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008020070 con 0x55f11e30f810 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fcffdffb700 1 -- 172.21.15.77:0/836422764 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd0080a9a20 con 0x55f11e30f810 2024-05-27T08:37:56.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 1 -- 172.21.15.77:0/836422764 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd00c00be60 con 0x55f11e30f810 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fcffdffb700 1 -- 172.21.15.77:0/836422764 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd00800f890 con 0x55f11e30f810 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 10 client.15102 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 1 -- 172.21.15.77:0/836422764 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f11e3953c0 con 0x55f11e30f810 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fcffdffb700 1 -- 172.21.15.77:0/836422764 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd00c00be60 con 0x55f11e30f810 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fcffdffb700 1 -- 172.21.15.77:0/836422764 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd0080ca020 con 0x55f11e30f810 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 10 client.15102 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 10 client.15102 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 20 client.15102 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55f11e399d80 0x55f11e398a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 4 client.15102 mds_command: new command op to 24410 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd0277ceb80 1 -- 172.21.15.77:0/836422764 --> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] -- command(tid 0: {"prefix": "status"}) -- 0x55f11e270b90 con 0x55f11e399d80 2024-05-27T08:37:56.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.509+0000 7fd015d9b700 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55f11e399d80 0x55f11e398a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fcffdffb700 10 client.15102 ms_handle_connect on v2:172.21.15.92:6836/2920087591 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd015d9b700 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55f11e399d80 0x55f11e398a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44bc96fadfc72103 server_cookie=9f1fb7f1ba4e70b8 in_seq=0 out_seq=0 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fcffdffb700 1 -- 172.21.15.77:0/836422764 <== mds.4 v2:172.21.15.92:6836/2920087591 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f11e270b90 con 0x55f11e399d80 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fcffdffb700 10 client.15102 handle_command_reply: tid=0 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 client.15102 shutdown 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 -- 172.21.15.77:0/836422764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 msgr2=0x55f11e333b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e333b20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0080a8f60 tx=0x7fd0080ca3b0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 -- 172.21.15.77:0/836422764 shutdown_connections 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f11e313f10 0x55f11e32ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x55f11e399d80 0x55f11e398a90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f11e30f810 0x55f11e333b20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 --2- 172.21.15.77:0/836422764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f11e3101c0 0x55f11e32e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 -- 172.21.15.77:0/836422764 >> 172.21.15.77:0/836422764 conn(0x55f11e23d030 msgr2=0x55f11e319df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fcffdffb700 0 client.15102 ms_handle_reset on v2:172.21.15.92:6836/2920087591 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 -- 172.21.15.77:0/836422764 shutdown_connections 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 1 -- 172.21.15.77:0/836422764 wait complete. 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.510+0000 7fd0277ceb80 20 client.15102 trim_cache size 0 max 16384 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24410, 2024-05-27T08:37:56.518 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:37:56.519 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:37:56.519 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:37:56.519 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 153.24665535700001, 2024-05-27T08:37:56.519 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:37:56.519 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:37:56.519 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:37:56.519 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 195.737338902 2024-05-27T08:37:56.519 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 -- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd0000b2900 msgr2=0x7fd0000b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd0000b2900 0x7fd0000b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00b5c0 tx=0x7fd00c005960 comp rx=0 tx=0).stop 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 -- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806bce0 msgr2=0x7fd01819be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806bce0 0x7fd01819be60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd00800d950 tx=0x7fd00800dc60 comp rx=0 tx=0).stop 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 -- 172.21.15.77:0/2302357343 shutdown_connections 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd01810fff0 0x7fd0181a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd01806bce0 0x7fd01819be60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd0000b2900 0x7fd0000b4dc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 --2- 172.21.15.77:0/2302357343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd01806c600 0x7fd01819c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:56.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 -- 172.21.15.77:0/2302357343 >> 172.21.15.77:0/2302357343 conn(0x7fd0180793f0 msgr2=0x7fd018111500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:56.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 -- 172.21.15.77:0/2302357343 shutdown_connections 2024-05-27T08:37:56.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:56.511+0000 7fd01cf38700 1 -- 172.21.15.77:0/2302357343 wait complete. 2024-05-27T08:37:57.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:56 smithi179 ceph-mon[36481]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:57.121 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:56 smithi077 ceph-mon[32007]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:57.171 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-05-27T08:37:57.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:56 smithi092 ceph-mon[36049]: pgmap v374: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:37:57.417 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:37:58.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.943+0000 7fc84e278700 1 -- 172.21.15.77:0/3029593462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc84806c690 msgr2=0x7fc8481103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.943+0000 7fc84e278700 1 --2- 172.21.15.77:0/3029593462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc84806c690 0x7fc8481103d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc84000b0a0 tx=0x7fc84000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 -- 172.21.15.77:0/3029593462 shutdown_connections 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 --2- 172.21.15.77:0/3029593462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc84806c690 0x7fc8481103d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 --2- 172.21.15.77:0/3029593462 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8481095b0 0x7fc848109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 --2- 172.21.15.77:0/3029593462 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84806bcd0 0x7fc84806c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 -- 172.21.15.77:0/3029593462 >> 172.21.15.77:0/3029593462 conn(0x7fc8480793f0 msgr2=0x7fc848079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 -- 172.21.15.77:0/3029593462 shutdown_connections 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 -- 172.21.15.77:0/3029593462 wait complete. 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 Processor -- start 2024-05-27T08:37:58.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.944+0000 7fc84e278700 1 -- start start 2024-05-27T08:37:58.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84e278700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84806bcd0 0x7fc848102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84e278700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc84806c690 0x7fc848102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84e278700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8481095b0 0x7fc848102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84e278700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481137c0 con 0x7fc8481095b0 2024-05-27T08:37:58.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84e278700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc848113640 con 0x7fc84806bcd0 2024-05-27T08:37:58.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84e278700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc848113940 con 0x7fc84806c690 2024-05-27T08:37:58.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8481095b0 0x7fc848102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8481095b0 0x7fc848102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59624/0 (socket says 172.21.15.77:59624) 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 -- 172.21.15.77:0/356858523 learned_addr learned my addr 172.21.15.77:0/356858523 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc8477fe700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc84806c690 0x7fc848102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc847fff700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84806bcd0 0x7fc848102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 -- 172.21.15.77:0/356858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc84806c690 msgr2=0x7fc848102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc84806c690 0x7fc848102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 -- 172.21.15.77:0/356858523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84806bcd0 msgr2=0x7fc848102510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84806bcd0 0x7fc848102510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 -- 172.21.15.77:0/356858523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8481a82e0 con 0x7fc8481095b0 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc84c815700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8481095b0 0x7fc848102f90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc8400062a0 tx=0x7fc840004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.945+0000 7fc8457fa700 1 -- 172.21.15.77:0/356858523 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840021070 con 0x7fc8481095b0 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.946+0000 7fc8457fa700 1 -- 172.21.15.77:0/356858523 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc840007c80 con 0x7fc8481095b0 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.946+0000 7fc84e278700 1 -- 172.21.15.77:0/356858523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8481a8570 con 0x7fc8481095b0 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.946+0000 7fc8457fa700 1 -- 172.21.15.77:0/356858523 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84001d450 con 0x7fc8481095b0 2024-05-27T08:37:58.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.946+0000 7fc84e278700 1 -- 172.21.15.77:0/356858523 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a8a20 con 0x7fc8481095b0 2024-05-27T08:37:58.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.948+0000 7fc8457fa700 1 -- 172.21.15.77:0/356858523 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc840004020 con 0x7fc8481095b0 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc8457fa700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8300b2880 0x7fc8300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc847fff700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8300b2880 0x7fc8300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc8457fa700 1 -- 172.21.15.77:0/356858523 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc84000f020 con 0x7fc8481095b0 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc847fff700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8300b2880 0x7fc8300b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc838006fd0 tx=0x7fc838008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc858b0eb80 1 Processor -- start 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc858b0eb80 1 -- start start 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e6ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 0x563fa3e73350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563fa3dcdc40 con 0x563fa3e6e810 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563fa3dbd7e0 con 0x563fa3e6f1c0 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563fa3dc1820 con 0x563fa3e72f10 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.949+0000 7fc847fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e6ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc847fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e6ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59626/0 (socket says 172.21.15.77:59626) 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc847fff700 1 -- 172.21.15.77:0/1691220243 learned_addr learned my addr 172.21.15.77:0/1691220243 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc8477fe700 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc84c815700 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 0x563fa3e73350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc847fff700 1 -- 172.21.15.77:0/1691220243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 msgr2=0x563fa3e73350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc847fff700 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 0x563fa3e73350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc847fff700 1 -- 172.21.15.77:0/1691220243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 msgr2=0x563fa3e728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc847fff700 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc847fff700 1 -- 172.21.15.77:0/1691220243 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa3d0d880 con 0x563fa3e6e810 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc847fff700 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e6ebf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d750 tx=0x7fc838010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2cf86ec62474f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc82e7fc700 1 -- 172.21.15.77:0/1691220243 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838024070 con 0x563fa3e6e810 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc82e7fc700 1 -- 172.21.15.77:0/1691220243 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc838014af0 con 0x563fa3e6e810 2024-05-27T08:37:58.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.950+0000 7fc82e7fc700 1 -- 172.21.15.77:0/1691220243 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83801fb60 con 0x563fa3e6e810 2024-05-27T08:37:58.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 -- 172.21.15.77:0/1691220243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 msgr2=0x563fa3e6ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e6ebf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d750 tx=0x7fc838010000 comp rx=0 tx=0).stop 2024-05-27T08:37:58.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 -- 172.21.15.77:0/1691220243 shutdown_connections 2024-05-27T08:37:58.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 0x563fa3e73350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e6ebf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/1691220243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 -- 172.21.15.77:0/1691220243 >> 172.21.15.77:0/1691220243 conn(0x563fa3d9c030 msgr2=0x563fa3e68e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:58.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 -- 172.21.15.77:0/1691220243 shutdown_connections 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 -- 172.21.15.77:0/1691220243 wait complete. 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.951+0000 7fc858b0eb80 1 -- start start 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc858b0eb80 1 Processor -- start 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e7a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 0x563fa3e77640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e72f10 0x563fa3e77b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563fa3cc3f50 con 0x563fa3e6e810 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563fa3d5f770 con 0x563fa3e72f10 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563fa3e7b0d0 con 0x563fa3e6f1c0 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc858b0eb80 10 client.0 _ll_register_callbacks cb 0x563fa3d588d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc847fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e7a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.958 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc847fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e7a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59638/0 (socket says 172.21.15.77:59638) 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc847fff700 1 -- 172.21.15.77:0/602793418 learned_addr learned my addr 172.21.15.77:0/602793418 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc84c815700 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e72f10 0x563fa3e77b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc8477fe700 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 0x563fa3e77640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc847fff700 1 -- 172.21.15.77:0/602793418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 msgr2=0x563fa3e77640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc847fff700 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 0x563fa3e77640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc847fff700 1 -- 172.21.15.77:0/602793418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e72f10 msgr2=0x563fa3e77b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc847fff700 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e72f10 0x563fa3e77b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc847fff700 1 -- 172.21.15.77:0/602793418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa3d5c640 con 0x563fa3e6e810 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.952+0000 7fc84c815700 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e72f10 0x563fa3e77b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc847fff700 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e7a340 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d440 tx=0x7fc838014920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4fd09f84d4a06f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc8277fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc8277fe700 1 -- 172.21.15.77:0/602793418 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83802e020 con 0x563fa3e6e810 2024-05-27T08:37:58.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc8277fe700 1 -- 172.21.15.77:0/602793418 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc83801f7c0 con 0x563fa3e6e810 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 1 -- 172.21.15.77:0/602793418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc838014af0 con 0x563fa3e6e810 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc8277fe700 1 -- 172.21.15.77:0/602793418 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc838024040 con 0x563fa3e6e810 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 10 client.15114 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 1 -- 172.21.15.77:0/602793418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fa3e857b0 con 0x563fa3e6e810 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc8277fe700 1 -- 172.21.15.77:0/602793418 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc838014af0 con 0x563fa3e6e810 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc8277fe700 1 -- 172.21.15.77:0/602793418 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc838027030 con 0x563fa3e6e810 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 10 client.15114 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 10 client.15114 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 20 client.15114 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x563fa3e87b80 0x563fa3e88ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 4 client.15114 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.953+0000 7fc858b0eb80 1 -- 172.21.15.77:0/602793418 --> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fa3e3a990 con 0x563fa3e87b80 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.954+0000 7fc8477fe700 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x563fa3e87b80 0x563fa3e88ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.954+0000 7fc8477fe700 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x563fa3e87b80 0x563fa3e88ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1fafb98814f1b0c server_cookie=d90eb20b435ec8f6 in_seq=0 out_seq=0 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.954+0000 7fc8277fe700 10 client.15114 ms_handle_connect on v2:172.21.15.179:6834/2178851321 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.955+0000 7fc8277fe700 1 -- 172.21.15.77:0/602793418 <== mds.3 v2:172.21.15.179:6834/2178851321 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563fa3e3a990 con 0x563fa3e87b80 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.955+0000 7fc8277fe700 10 client.15114 handle_command_reply: tid=0 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.955+0000 7fc858b0eb80 1 client.15114 shutdown 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 -- 172.21.15.77:0/602793418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 msgr2=0x563fa3e7a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e7a340 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d440 tx=0x7fc838014920 comp rx=0 tx=0).stop 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 -- 172.21.15.77:0/602793418 shutdown_connections 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x563fa3e87b80 0x563fa3e88ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e72f10 0x563fa3e77b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc8277fe700 0 client.15114 ms_handle_reset on v2:172.21.15.179:6834/2178851321 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e7a340 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/602793418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 0x563fa3e77640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 -- 172.21.15.77:0/602793418 >> 172.21.15.77:0/602793418 conn(0x563fa3d9c030 msgr2=0x563fa3e684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:58.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 -- 172.21.15.77:0/602793418 shutdown_connections 2024-05-27T08:37:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 1 -- 172.21.15.77:0/602793418 wait complete. 2024-05-27T08:37:58.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.956+0000 7fc858b0eb80 20 client.15114 trim_cache size 0 max 16384 2024-05-27T08:37:58.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.976+0000 7fc858b0eb80 1 Processor -- start 2024-05-27T08:37:58.978 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:37:58 smithi077 ceph-mon[32007]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:58.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.976+0000 7fc858b0eb80 1 -- start start 2024-05-27T08:37:58.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6e810 0x563fa3e68c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 0x563fa3e69180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e72f10 0x563fa3e92450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563fa3e87330 con 0x563fa3e72f10 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563fa3e87a00 con 0x563fa3e6e810 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563fa3e880d0 con 0x563fa3e6f1c0 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e72f10 0x563fa3e92450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e72f10 0x563fa3e92450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59642/0 (socket says 172.21.15.77:59642) 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 -- 172.21.15.77:0/2669738408 learned_addr learned my addr 172.21.15.77:0/2669738408 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc847fff700 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6e810 0x563fa3e68c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 -- 172.21.15.77:0/2669738408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 msgr2=0x563fa3e69180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 0x563fa3e69180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 -- 172.21.15.77:0/2669738408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6e810 msgr2=0x563fa3e68c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6e810 0x563fa3e68c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 -- 172.21.15.77:0/2669738408 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fa3e84370 con 0x563fa3e72f10 2024-05-27T08:37:58.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc847fff700 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6e810 0x563fa3e68c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc84c815700 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e72f10 0x563fa3e92450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc8400e1440 tx=0x7fc8400099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b91a558dfdbd35 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc82dffb700 1 -- 172.21.15.77:0/2669738408 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400aaff0 con 0x563fa3e72f10 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc82dffb700 1 -- 172.21.15.77:0/2669738408 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8400af3a0 con 0x563fa3e72f10 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.977+0000 7fc82dffb700 1 -- 172.21.15.77:0/2669738408 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400b3b50 con 0x563fa3e72f10 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 -- 172.21.15.77:0/2669738408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e72f10 msgr2=0x563fa3e92450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e72f10 0x563fa3e92450 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc8400e1440 tx=0x7fc8400099e0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 -- 172.21.15.77:0/2669738408 shutdown_connections 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e72f10 0x563fa3e92450 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6e810 0x563fa3e68c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/2669738408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e6f1c0 0x563fa3e69180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 -- 172.21.15.77:0/2669738408 >> 172.21.15.77:0/2669738408 conn(0x563fa3d9c030 msgr2=0x563fa3e86b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 -- 172.21.15.77:0/2669738408 shutdown_connections 2024-05-27T08:37:58.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 -- 172.21.15.77:0/2669738408 wait complete. 2024-05-27T08:37:58.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.978+0000 7fc858b0eb80 1 -- start start 2024-05-27T08:37:58.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc858b0eb80 1 Processor -- start 2024-05-27T08:37:58.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e92ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc858b0eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 0x563fa3e8e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563fa3e88e70 con 0x563fa3e6e810 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x563fa3e887a0 con 0x563fa3e6f1c0 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc858b0eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563fa3e89540 con 0x563fa3e72f10 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc858b0eb80 10 client.0 _ll_register_callbacks cb 0x563fa3d588d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc847fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc847fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59644/0 (socket says 172.21.15.77:59644) 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc847fff700 1 -- 172.21.15.77:0/4146519884 learned_addr learned my addr 172.21.15.77:0/4146519884 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc84c815700 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 0x563fa3e8e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc8477fe700 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e92ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc847fff700 1 -- 172.21.15.77:0/4146519884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 msgr2=0x563fa3e8e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:58.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc847fff700 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 0x563fa3e8e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc847fff700 1 -- 172.21.15.77:0/4146519884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 msgr2=0x563fa3e92ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc847fff700 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e92ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc847fff700 1 -- 172.21.15.77:0/4146519884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fa3e8f1e0 con 0x563fa3e6e810 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.979+0000 7fc8477fe700 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e92ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc847fff700 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e92580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc838008010 tx=0x7fc83802a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a376c66907e77bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc8277fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc8277fe700 1 -- 172.21.15.77:0/4146519884 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83802e070 con 0x563fa3e6e810 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc8277fe700 1 -- 172.21.15.77:0/4146519884 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fc83801f6f0 con 0x563fa3e6e810 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 1 -- 172.21.15.77:0/4146519884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8400af3a0 con 0x563fa3e6e810 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc8277fe700 1 -- 172.21.15.77:0/4146519884 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc838005a50 con 0x563fa3e6e810 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 10 client.15120 fetch_fsmap learned FSMap version 28 2024-05-27T08:37:58.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 1 -- 172.21.15.77:0/4146519884 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fa3ef3e70 con 0x563fa3e6e810 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc8277fe700 1 -- 172.21.15.77:0/4146519884 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8400af3a0 con 0x563fa3e6e810 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc8277fe700 1 -- 172.21.15.77:0/4146519884 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc838027040 con 0x563fa3e6e810 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 10 client.15120 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 10 client.15120 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 20 client.15120 populate_metadata read hostname 'smithi077' 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x563fa3e87b80 0x563fa3f087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 4 client.15120 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.980+0000 7fc858b0eb80 1 -- 172.21.15.77:0/4146519884 --> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] -- command(tid 0: {"prefix": "status"}) -- 0x563fa3dcfb90 con 0x563fa3e87b80 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.981+0000 7fc8477fe700 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x563fa3e87b80 0x563fa3f087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.981+0000 7fc8477fe700 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x563fa3e87b80 0x563fa3f087d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef631fdcebfbe6c server_cookie=fae603a0ba5c4309 in_seq=0 out_seq=0 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.981+0000 7fc8277fe700 10 client.15120 ms_handle_connect on v2:172.21.15.179:6834/2178851321 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.981+0000 7fc8277fe700 1 -- 172.21.15.77:0/4146519884 <== mds.3 v2:172.21.15.179:6834/2178851321 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563fa3dcfb90 con 0x563fa3e87b80 2024-05-27T08:37:58.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.981+0000 7fc8277fe700 10 client.15120 handle_command_reply: tid=0 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.981+0000 7fc858b0eb80 1 client.15120 shutdown 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.981+0000 7fc858b0eb80 1 -- 172.21.15.77:0/4146519884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 msgr2=0x563fa3e92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.981+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e92580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc838008010 tx=0x7fc83802a700 comp rx=0 tx=0).stop 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 1 -- 172.21.15.77:0/4146519884 shutdown_connections 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fa3e72f10 0x563fa3e8e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563fa3e6e810 0x563fa3e92580 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x563fa3e87b80 0x563fa3f087d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc8277fe700 0 client.15120 ms_handle_reset on v2:172.21.15.179:6834/2178851321 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 1 --2- 172.21.15.77:0/4146519884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563fa3e6f1c0 0x563fa3e92ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 1 -- 172.21.15.77:0/4146519884 >> 172.21.15.77:0/4146519884 conn(0x563fa3d9c030 msgr2=0x563fa3e6d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 1 -- 172.21.15.77:0/4146519884 shutdown_connections 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 1 -- 172.21.15.77:0/4146519884 wait complete. 2024-05-27T08:37:58.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc858b0eb80 20 client.15120 trim_cache size 0 max 16384 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24412, 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 156.73326499300001, 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:37:58.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:37:58.990 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 203.33057935100001 2024-05-27T08:37:58.990 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:37:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc84e278700 1 -- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8300b2880 msgr2=0x7fc8300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc84e278700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8300b2880 0x7fc8300b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc838006fd0 tx=0x7fc838008040 comp rx=0 tx=0).stop 2024-05-27T08:37:58.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc84e278700 1 -- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8481095b0 msgr2=0x7fc848102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.982+0000 7fc84e278700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8481095b0 0x7fc848102f90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc8400062a0 tx=0x7fc840004e20 comp rx=0 tx=0).stop 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.983+0000 7fc84e278700 1 -- 172.21.15.77:0/356858523 shutdown_connections 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.983+0000 7fc84e278700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8481095b0 0x7fc848102f90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.983+0000 7fc84e278700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc84806bcd0 0x7fc848102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.983+0000 7fc84e278700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fc8300b2880 0x7fc8300b4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.983+0000 7fc84e278700 1 --2- 172.21.15.77:0/356858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc84806c690 0x7fc848102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.983+0000 7fc84e278700 1 -- 172.21.15.77:0/356858523 >> 172.21.15.77:0/356858523 conn(0x7fc8480793f0 msgr2=0x7fc848101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.983+0000 7fc84e278700 1 -- 172.21.15.77:0/356858523 shutdown_connections 2024-05-27T08:37:58.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:37:58.983+0000 7fc84e278700 1 -- 172.21.15.77:0/356858523 wait complete. 2024-05-27T08:37:59.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:37:58 smithi179 ceph-mon[36481]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:59.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:37:58 smithi092 ceph-mon[36049]: pgmap v375: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:37:59.535 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-05-27T08:37:59.773 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:00.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:00 smithi077 ceph-mon[32007]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:38:01.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:00 smithi179 ceph-mon[36481]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:38:01.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:00 smithi092 ceph-mon[36049]: pgmap v376: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-27T08:38:01.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.545+0000 7f17e83d0700 1 -- 172.21.15.77:0/3346294453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 msgr2=0x7f17e0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.545+0000 7f17e83d0700 1 --2- 172.21.15.77:0/3346294453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 0x7f17e0108900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009990 tx=0x7f17dc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.545+0000 7f17e83d0700 1 -- 172.21.15.77:0/3346294453 shutdown_connections 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.545+0000 7f17e83d0700 1 --2- 172.21.15.77:0/3346294453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17e0102e00 0x7f17e010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.545+0000 7f17e83d0700 1 --2- 172.21.15.77:0/3346294453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 0x7f17e0108900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.545+0000 7f17e83d0700 1 --2- 172.21.15.77:0/3346294453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17e0102440 0x7f17e01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.545+0000 7f17e83d0700 1 -- 172.21.15.77:0/3346294453 >> 172.21.15.77:0/3346294453 conn(0x7f17e00793f0 msgr2=0x7f17e0079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 -- 172.21.15.77:0/3346294453 shutdown_connections 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 -- 172.21.15.77:0/3346294453 wait complete. 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 Processor -- start 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 -- start start 2024-05-27T08:38:01.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17e0102440 0x7f17e019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17e0102e00 0x7f17e019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 0x7f17e01a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0112650 con 0x7f17e0108520 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01124d0 con 0x7f17e0102e00 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e83d0700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f17e01127d0 con 0x7f17e0102440 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.546+0000 7f17e696d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 0x7f17e01a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e696d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 0x7f17e01a5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33622/0 (socket says 172.21.15.77:33622) 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e696d700 1 -- 172.21.15.77:0/2687046672 learned_addr learned my addr 172.21.15.77:0/2687046672 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e596b700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17e0102e00 0x7f17e019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e616c700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17e0102440 0x7f17e019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e696d700 1 -- 172.21.15.77:0/2687046672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17e0102440 msgr2=0x7f17e019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e696d700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17e0102440 0x7f17e019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e696d700 1 -- 172.21.15.77:0/2687046672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17e0102e00 msgr2=0x7f17e019b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e696d700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17e0102e00 0x7f17e019b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e696d700 1 -- 172.21.15.77:0/2687046672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e01a6090 con 0x7f17e0108520 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e696d700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 0x7f17e01a5b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17d800b970 tx=0x7f17d800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17d77fe700 1 -- 172.21.15.77:0/2687046672 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d800d860 con 0x7f17e0108520 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e83d0700 1 -- 172.21.15.77:0/2687046672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e01a62c0 con 0x7f17e0108520 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17d77fe700 1 -- 172.21.15.77:0/2687046672 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d800e430 con 0x7f17e0108520 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17d77fe700 1 -- 172.21.15.77:0/2687046672 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8017660 con 0x7f17e0108520 2024-05-27T08:38:01.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.547+0000 7f17e83d0700 1 -- 172.21.15.77:0/2687046672 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e01a6780 con 0x7f17e0108520 2024-05-27T08:38:01.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.550+0000 7f17d77fe700 1 -- 172.21.15.77:0/2687046672 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f17d800da00 con 0x7f17e0108520 2024-05-27T08:38:01.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.550+0000 7f17d77fe700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f17cc0b2880 0x7f17cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.550+0000 7f17e616c700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f17cc0b2880 0x7f17cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17d77fe700 1 -- 172.21.15.77:0/2687046672 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f17d80e1e30 con 0x7f17e0108520 2024-05-27T08:38:01.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17f2c66b80 1 Processor -- start 2024-05-27T08:38:01.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17f2c66b80 1 -- start start 2024-05-27T08:38:01.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17e616c700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f17cc0b2880 0x7f17cc0b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00b5c0 tx=0x7f17dc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:01.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b222d0 0x562604b226b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b1c850 0x562604b1ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1d210 0x562604b29030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562604a7ad20 con 0x562604b1c850 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562604a6a8c0 con 0x562604b222d0 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562604a6e900 con 0x562604b1d210 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17e596b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b1c850 0x562604b1ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17e596b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b1c850 0x562604b1ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33624/0 (socket says 172.21.15.77:33624) 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17e596b700 1 -- 172.21.15.77:0/960703050 learned_addr learned my addr 172.21.15.77:0/960703050 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17e596b700 1 -- 172.21.15.77:0/960703050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1d210 msgr2=0x562604b29030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.551+0000 7f17e596b700 1 --2- 172.21.15.77:0/960703050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1d210 0x562604b29030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.552+0000 7f17e596b700 1 -- 172.21.15.77:0/960703050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b222d0 msgr2=0x562604b226b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.552+0000 7f17e596b700 1 --2- 172.21.15.77:0/960703050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b222d0 0x562604b226b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.552+0000 7f17e596b700 1 -- 172.21.15.77:0/960703050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562604a13f60 con 0x562604b1c850 2024-05-27T08:38:01.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.552+0000 7f17e596b700 1 --2- 172.21.15.77:0/960703050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b1c850 0x562604b1ccd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17d000bcf0 tx=0x7f17d00103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461d2d3ab29550ed server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:01.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.552+0000 7f17d4d70700 1 -- 172.21.15.77:0/960703050 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0010dc0 con 0x562604b1c850 2024-05-27T08:38:01.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.552+0000 7f17d4d70700 1 -- 172.21.15.77:0/960703050 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d00044e0 con 0x562604b1c850 2024-05-27T08:38:01.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.552+0000 7f17d4d70700 1 -- 172.21.15.77:0/960703050 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0005350 con 0x562604b1c850 2024-05-27T08:38:01.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- 172.21.15.77:0/960703050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b1c850 msgr2=0x562604b1ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/960703050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b1c850 0x562604b1ccd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17d000bcf0 tx=0x7f17d00103d0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- 172.21.15.77:0/960703050 shutdown_connections 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/960703050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1d210 0x562604b29030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/960703050 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b222d0 0x562604b226b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/960703050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b1c850 0x562604b1ccd0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- 172.21.15.77:0/960703050 >> 172.21.15.77:0/960703050 conn(0x562604a481f0 msgr2=0x562604b17390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- 172.21.15.77:0/960703050 shutdown_connections 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- 172.21.15.77:0/960703050 wait complete. 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- start start 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 Processor -- start 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b28e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b26600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.559 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562604970f50 con 0x562604b222d0 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5626049d2d50 con 0x562604b1d210 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562604b29690 con 0x562604b1c850 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.553+0000 7f17f2c66b80 10 client.0 _ll_register_callbacks cb 0x562604a05900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b26600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b26600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33640/0 (socket says 172.21.15.77:33640) 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e616c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b28e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 -- 172.21.15.77:0/2255988721 learned_addr learned my addr 172.21.15.77:0/2255988721 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e596b700 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 -- 172.21.15.77:0/2255988721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 msgr2=0x562604b28e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b28e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 -- 172.21.15.77:0/2255988721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 msgr2=0x562604b260c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b260c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 -- 172.21.15.77:0/2255988721 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626049ba880 con 0x562604b222d0 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17e696d700 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b26600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f17d80a93e0 tx=0x7f17d80a90a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199f0c58cb4b78a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17c9ffb700 1 -- 172.21.15.77:0/2255988721 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d80c3060 con 0x562604b222d0 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17c9ffb700 1 -- 172.21.15.77:0/2255988721 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f17d800e8e0 con 0x562604b222d0 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17f2c66b80 1 -- 172.21.15.77:0/2255988721 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d00044e0 con 0x562604b222d0 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17c9ffb700 1 -- 172.21.15.77:0/2255988721 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17d80ca050 con 0x562604b222d0 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17f2c66b80 10 client.15132 fetch_fsmap learned FSMap version 28 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17f2c66b80 1 -- 172.21.15.77:0/2255988721 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562604a24710 con 0x562604b222d0 2024-05-27T08:38:01.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.554+0000 7f17c9ffb700 1 -- 172.21.15.77:0/2255988721 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d00044e0 con 0x562604b222d0 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17c9ffb700 1 -- 172.21.15.77:0/2255988721 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f17d801f030 con 0x562604b222d0 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17f2c66b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17f2c66b80 10 client.15132 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17f2c66b80 20 client.15132 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x562604b36240 0x562604b376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17f2c66b80 4 client.15132 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17f2c66b80 1 -- 172.21.15.77:0/2255988721 --> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562604a7cc70 con 0x562604b36240 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17e616c700 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x562604b36240 0x562604b376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17c9ffb700 10 client.15132 ms_handle_connect on v2:172.21.15.179:6836/2857088493 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.555+0000 7f17e616c700 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x562604b36240 0x562604b376b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4255a2c2ff39c57 server_cookie=61830551bb99ea53 in_seq=0 out_seq=0 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17c9ffb700 1 -- 172.21.15.77:0/2255988721 <== mds.2 v2:172.21.15.179:6836/2857088493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562604a7cc70 con 0x562604b36240 2024-05-27T08:38:01.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17c9ffb700 10 client.15132 handle_command_reply: tid=0 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 client.15132 shutdown 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 -- 172.21.15.77:0/2255988721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 msgr2=0x562604b26600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b26600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f17d80a93e0 tx=0x7f17d80a90a0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 -- 172.21.15.77:0/2255988721 shutdown_connections 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b26600 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x562604b36240 0x562604b376b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b28e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17c9ffb700 0 client.15132 ms_handle_reset on v2:172.21.15.179:6836/2857088493 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/2255988721 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b260c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.556+0000 7f17f2c66b80 1 -- 172.21.15.77:0/2255988721 >> 172.21.15.77:0/2255988721 conn(0x562604a481f0 msgr2=0x562604b16aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.557+0000 7f17f2c66b80 1 -- 172.21.15.77:0/2255988721 shutdown_connections 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.557+0000 7f17f2c66b80 1 -- 172.21.15.77:0/2255988721 wait complete. 2024-05-27T08:38:01.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.557+0000 7f17f2c66b80 20 client.15132 trim_cache size 0 max 16384 2024-05-27T08:38:01.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.577+0000 7f17f2c66b80 1 Processor -- start 2024-05-27T08:38:01.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.577+0000 7f17f2c66b80 1 -- start start 2024-05-27T08:38:01.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b35b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562604b359f0 con 0x562604b222d0 2024-05-27T08:38:01.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562604b360c0 con 0x562604b1d210 2024-05-27T08:38:01.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562604b36790 con 0x562604b1c850 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b278f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33644/0 (socket says 172.21.15.77:33644) 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 -- 172.21.15.77:0/140630316 learned_addr learned my addr 172.21.15.77:0/140630316 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e596b700 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e616c700 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b35b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 -- 172.21.15.77:0/140630316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 msgr2=0x562604b35b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b35b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 -- 172.21.15.77:0/140630316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 msgr2=0x562604b351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 -- 172.21.15.77:0/140630316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562604b24630 con 0x562604b222d0 2024-05-27T08:38:01.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17e696d700 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b278f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17d80a9610 tx=0x7f17d80911a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7b06daa414285a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17cbfff700 1 -- 172.21.15.77:0/140630316 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d80ad800 con 0x562604b222d0 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17cbfff700 1 -- 172.21.15.77:0/140630316 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d80b22b0 con 0x562604b222d0 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.578+0000 7f17cbfff700 1 -- 172.21.15.77:0/140630316 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d80a3b70 con 0x562604b222d0 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 -- 172.21.15.77:0/140630316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 msgr2=0x562604b278f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b278f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f17d80a9610 tx=0x7f17d80911a0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 -- 172.21.15.77:0/140630316 shutdown_connections 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b278f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b35b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/140630316 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 -- 172.21.15.77:0/140630316 >> 172.21.15.77:0/140630316 conn(0x562604a481f0 msgr2=0x562604b13710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 -- 172.21.15.77:0/140630316 shutdown_connections 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 -- 172.21.15.77:0/140630316 wait complete. 2024-05-27T08:38:01.586 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 -- start start 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.579+0000 7f17f2c66b80 1 Processor -- start 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b40b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b41070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b39ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562604b37530 con 0x562604b222d0 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562604b36e60 con 0x562604b1d210 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562604b37c00 con 0x562604b1c850 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 10 client.0 _ll_register_callbacks cb 0x7f17d80a8e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b39ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b39ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33652/0 (socket says 172.21.15.77:33652) 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 -- 172.21.15.77:0/3073309890 learned_addr learned my addr 172.21.15.77:0/3073309890 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e596b700 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b41070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e616c700 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b40b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 -- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 msgr2=0x562604b40b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b40b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 -- 172.21.15.77:0/3073309890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 msgr2=0x562604b41070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b41070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 -- 172.21.15.77:0/3073309890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562604b3a010 con 0x562604b222d0 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e616c700 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b40b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17e696d700 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b39ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f17d800b940 tx=0x7f17d80a9020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45cf43813d9f5f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17c9ffb700 1 -- 172.21.15.77:0/3073309890 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8087020 con 0x562604b222d0 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17c9ffb700 1 -- 172.21.15.77:0/3073309890 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f17d800e8e0 con 0x562604b222d0 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 1 -- 172.21.15.77:0/3073309890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17d80b22b0 con 0x562604b222d0 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17c9ffb700 1 -- 172.21.15.77:0/3073309890 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17d80a3b70 con 0x562604b222d0 2024-05-27T08:38:01.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 10 client.15138 fetch_fsmap learned FSMap version 28 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.580+0000 7f17f2c66b80 1 -- 172.21.15.77:0/3073309890 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562604ba22d0 con 0x562604b222d0 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17c9ffb700 1 -- 172.21.15.77:0/3073309890 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17d80b22b0 con 0x562604b222d0 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17c9ffb700 1 -- 172.21.15.77:0/3073309890 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f17d808c070 con 0x562604b222d0 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17f2c66b80 10 client.15138 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17f2c66b80 10 client.15138 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17f2c66b80 20 client.15138 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x562604ba68c0 0x562604ba55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17f2c66b80 4 client.15138 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17f2c66b80 1 -- 172.21.15.77:0/3073309890 --> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] -- command(tid 0: {"prefix": "status"}) -- 0x562604a76450 con 0x562604ba68c0 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17e616c700 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x562604ba68c0 0x562604ba55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17c9ffb700 10 client.15138 ms_handle_connect on v2:172.21.15.179:6836/2857088493 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.581+0000 7f17e616c700 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x562604ba68c0 0x562604ba55d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d0c0d68d05a63dd server_cookie=b65f3b6bf2f2397a in_seq=0 out_seq=0 2024-05-27T08:38:01.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17c9ffb700 1 -- 172.21.15.77:0/3073309890 <== mds.2 v2:172.21.15.179:6836/2857088493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562604a76450 con 0x562604ba68c0 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17c9ffb700 10 client.15138 handle_command_reply: tid=0 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 client.15138 shutdown 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 -- 172.21.15.77:0/3073309890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 msgr2=0x562604b39ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b39ad0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f17d800b940 tx=0x7f17d80a9020 comp rx=0 tx=0).stop 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 -- 172.21.15.77:0/3073309890 shutdown_connections 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562604b222d0 0x562604b39ad0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x562604ba68c0 0x562604ba55d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562604b1c850 0x562604b40b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 --2- 172.21.15.77:0/3073309890 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562604b1d210 0x562604b41070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 -- 172.21.15.77:0/3073309890 >> 172.21.15.77:0/3073309890 conn(0x562604a481f0 msgr2=0x562604b351e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17c9ffb700 0 client.15138 ms_handle_reset on v2:172.21.15.179:6836/2857088493 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 -- 172.21.15.77:0/3073309890 shutdown_connections 2024-05-27T08:38:01.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 1 -- 172.21.15.77:0/3073309890 wait complete. 2024-05-27T08:38:01.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.582+0000 7f17f2c66b80 20 client.15138 trim_cache size 0 max 16384 2024-05-27T08:38:01.591 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:38:01.591 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24418, 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 160.33866804100001, 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 203.51269502299999 2024-05-27T08:38:01.592 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:38:01.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 -- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f17cc0b2880 msgr2=0x7f17cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f17cc0b2880 0x7f17cc0b4d40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00b5c0 tx=0x7f17dc005960 comp rx=0 tx=0).stop 2024-05-27T08:38:01.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 -- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 msgr2=0x7f17e01a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:01.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 0x7f17e01a5b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17d800b970 tx=0x7f17d800bc80 comp rx=0 tx=0).stop 2024-05-27T08:38:01.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 -- 172.21.15.77:0/2687046672 shutdown_connections 2024-05-27T08:38:01.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17e0108520 0x7f17e01a5b50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f17cc0b2880 0x7f17cc0b4d40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17e0102440 0x7f17e019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 --2- 172.21.15.77:0/2687046672 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f17e0102e00 0x7f17e019b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:01.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 -- 172.21.15.77:0/2687046672 >> 172.21.15.77:0/2687046672 conn(0x7f17e00793f0 msgr2=0x7f17e01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:01.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 -- 172.21.15.77:0/2687046672 shutdown_connections 2024-05-27T08:38:01.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:01.583+0000 7f17e83d0700 1 -- 172.21.15.77:0/2687046672 wait complete. 2024-05-27T08:38:02.182 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-05-27T08:38:02.421 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:02.850 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:02 smithi077 ceph-mon[32007]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:38:03.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:02 smithi179 ceph-mon[36481]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:38:03.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:02 smithi092 ceph-mon[36049]: pgmap v377: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-05-27T08:38:04.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 -- 172.21.15.77:0/2949048094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f4102470 msgr2=0x7f11f41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 --2- 172.21.15.77:0/2949048094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f4102470 0x7f11f41028f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009990 tx=0x7f11e8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 -- 172.21.15.77:0/2949048094 shutdown_connections 2024-05-27T08:38:04.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 --2- 172.21.15.77:0/2949048094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f4102e30 0x7f11f410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 --2- 172.21.15.77:0/2949048094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f41084b0 0x7f11f4108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.110 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 --2- 172.21.15.77:0/2949048094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f4102470 0x7f11f41028f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 -- 172.21.15.77:0/2949048094 >> 172.21.15.77:0/2949048094 conn(0x7f11f4079410 msgr2=0x7f11f4079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 -- 172.21.15.77:0/2949048094 shutdown_connections 2024-05-27T08:38:04.111 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.108+0000 7f11f901a700 1 -- 172.21.15.77:0/2949048094 wait complete. 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.109+0000 7f11f901a700 1 Processor -- start 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.109+0000 7f11f901a700 1 -- start start 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f901a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f4102470 0x7f11f4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f901a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f4102e30 0x7f11f4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f901a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f41084b0 0x7f11f4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f901a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11f41126c0 con 0x7f11f41084b0 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f901a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4112540 con 0x7f11f4102470 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f901a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f11f4112840 con 0x7f11f4102e30 2024-05-27T08:38:04.113 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f41084b0 0x7f11f4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f259c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f4102e30 0x7f11f4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f41084b0 0x7f11f4199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33666/0 (socket says 172.21.15.77:33666) 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 -- 172.21.15.77:0/1507290225 learned_addr learned my addr 172.21.15.77:0/1507290225 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f4102470 0x7f11f4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 -- 172.21.15.77:0/1507290225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f4102e30 msgr2=0x7f11f4198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f4102e30 0x7f11f4198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 -- 172.21.15.77:0/1507290225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f4102470 msgr2=0x7f11f4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f4102470 0x7f11f4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 -- 172.21.15.77:0/1507290225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f41a8300 con 0x7f11f41084b0 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f4102470 0x7f11f4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:04.114 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f359e700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f41084b0 0x7f11f4199510 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00b970 tx=0x7f11ec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11dbfff700 1 -- 172.21.15.77:0/1507290225 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec019070 con 0x7f11f41084b0 2024-05-27T08:38:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11dbfff700 1 -- 172.21.15.77:0/1507290225 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ec00cde0 con 0x7f11f41084b0 2024-05-27T08:38:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.110+0000 7f11f901a700 1 -- 172.21.15.77:0/1507290225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f41a85f0 con 0x7f11f41084b0 2024-05-27T08:38:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.111+0000 7f11dbfff700 1 -- 172.21.15.77:0/1507290225 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec007370 con 0x7f11f41084b0 2024-05-27T08:38:04.115 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.111+0000 7f11f901a700 1 -- 172.21.15.77:0/1507290225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f41a8b00 con 0x7f11f41084b0 2024-05-27T08:38:04.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.113+0000 7f11dbfff700 1 -- 172.21.15.77:0/1507290225 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f11ec01c070 con 0x7f11f41084b0 2024-05-27T08:38:04.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f11dbfff700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f11dc0b2880 0x7f11dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f11dc0b2880 0x7f11dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f11dbfff700 1 -- 172.21.15.77:0/1507290225 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f11ec0e1590 con 0x7f11f41084b0 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f11dc0b2880 0x7f11dc0b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11e40099d0 tx=0x7f11e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f12038b0b80 1 Processor -- start 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f12038b0b80 1 -- start start 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df76716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df766bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.114+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766c210 0x559df7678030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f12038b0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559df75c9d20 con 0x559df76712d0 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f12038b0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559df75b98c0 con 0x559df766b850 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f12038b0b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559df75bd900 con 0x559df766c210 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df76716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.122 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df76716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33670/0 (socket says 172.21.15.77:33670) 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 -- 172.21.15.77:0/1794535667 learned_addr learned my addr 172.21.15.77:0/1794535667 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f259c700 1 --2- 172.21.15.77:0/1794535667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df766bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 -- 172.21.15.77:0/1794535667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766c210 msgr2=0x559df7678030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1794535667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766c210 0x559df7678030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 -- 172.21.15.77:0/1794535667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 msgr2=0x559df766bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1794535667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df766bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 -- 172.21.15.77:0/1794535667 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df7562f60 con 0x559df76712d0 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1794535667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df76716b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f11e400d7c0 tx=0x7f11e4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf33ace016a3d18 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11d97fa700 1 -- 172.21.15.77:0/1794535667 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4024070 con 0x559df76712d0 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11d97fa700 1 -- 172.21.15.77:0/1794535667 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e4014af0 con 0x559df76712d0 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.115+0000 7f11d97fa700 1 -- 172.21.15.77:0/1794535667 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e401fc10 con 0x559df76712d0 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 -- 172.21.15.77:0/1794535667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 msgr2=0x559df76716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1794535667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df76716b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f11e400d7c0 tx=0x7f11e4010000 comp rx=0 tx=0).stop 2024-05-27T08:38:04.123 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 -- 172.21.15.77:0/1794535667 shutdown_connections 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1794535667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766c210 0x559df7678030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1794535667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df76716b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1794535667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df766bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 -- 172.21.15.77:0/1794535667 >> 172.21.15.77:0/1794535667 conn(0x559df75971f0 msgr2=0x559df7666390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 -- 172.21.15.77:0/1794535667 shutdown_connections 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 -- 172.21.15.77:0/1794535667 wait complete. 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 -- start start 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.116+0000 7f12038b0b80 1 Processor -- start 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766b850 0x559df7677d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 0x559df7675010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df7675550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f12038b0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559df74bff50 con 0x559df76712d0 2024-05-27T08:38:04.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f12038b0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559df7521d50 con 0x559df766c210 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f12038b0b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559df7678690 con 0x559df766b850 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f12038b0b80 10 client.0 _ll_register_callbacks cb 0x559df7554900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df7675550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df7675550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33682/0 (socket says 172.21.15.77:33682) 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 -- 172.21.15.77:0/1178090258 learned_addr learned my addr 172.21.15.77:0/1178090258 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f259c700 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 0x559df7675010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766b850 0x559df7677d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 -- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766b850 msgr2=0x559df7677d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766b850 0x559df7677d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 -- 172.21.15.77:0/1178090258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 msgr2=0x559df7675010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 0x559df7675010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 -- 172.21.15.77:0/1178090258 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df7509880 con 0x559df76712d0 2024-05-27T08:38:04.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766b850 0x559df7677d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11f359e700 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df7675550 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0a4c30 tx=0x7f11ec0b1420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e297f5854296943 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11d6ff5700 1 -- 172.21.15.77:0/1178090258 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0c9e60 con 0x559df76712d0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f11d6ff5700 1 -- 172.21.15.77:0/1178090258 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f11ec0a9020 con 0x559df76712d0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.117+0000 7f12038b0b80 1 -- 172.21.15.77:0/1178090258 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e4014af0 con 0x559df76712d0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f11d6ff5700 1 -- 172.21.15.77:0/1178090258 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f11ec0a8bb0 con 0x559df76712d0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f12038b0b80 10 client.15150 fetch_fsmap learned FSMap version 28 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f12038b0b80 1 -- 172.21.15.77:0/1178090258 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559df7573710 con 0x559df76712d0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f11d6ff5700 1 -- 172.21.15.77:0/1178090258 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e4014af0 con 0x559df76712d0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f11d6ff5700 1 -- 172.21.15.77:0/1178090258 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f11ec0e1110 con 0x559df76712d0 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f12038b0b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:38:04.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f12038b0b80 10 client.15150 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f12038b0b80 20 client.15150 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x559df7685180 0x559df76865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f12038b0b80 4 client.15150 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f12038b0b80 1 -- 172.21.15.77:0/1178090258 --> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559df75cbc70 con 0x559df7685180 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.118+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x559df7685180 0x559df76865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.119+0000 7f11d6ff5700 10 client.15150 ms_handle_connect on v2:172.21.15.179:6838/2672999131 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.119+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x559df7685180 0x559df76865f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be81fe0016907e40 server_cookie=e5241cd5b7355364 in_seq=0 out_seq=0 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.119+0000 7f11d6ff5700 1 -- 172.21.15.77:0/1178090258 <== mds.1 v2:172.21.15.179:6838/2672999131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559df75cbc70 con 0x559df7685180 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.119+0000 7f11d6ff5700 10 client.15150 handle_command_reply: tid=0 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 client.15150 shutdown 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 -- 172.21.15.77:0/1178090258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 msgr2=0x559df7675550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df7675550 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f11ec0a4c30 tx=0x7f11ec0b1420 comp rx=0 tx=0).stop 2024-05-27T08:38:04.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 -- 172.21.15.77:0/1178090258 shutdown_connections 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x559df7685180 0x559df76865f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df76712d0 0x559df7675550 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f11d6ff5700 0 client.15150 ms_handle_reset on v2:172.21.15.179:6838/2672999131 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df766b850 0x559df7677d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 --2- 172.21.15.77:0/1178090258 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 0x559df7675010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 -- 172.21.15.77:0/1178090258 >> 172.21.15.77:0/1178090258 conn(0x559df75971f0 msgr2=0x559df7665aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 -- 172.21.15.77:0/1178090258 shutdown_connections 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 1 -- 172.21.15.77:0/1178090258 wait complete. 2024-05-27T08:38:04.128 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.120+0000 7f12038b0b80 20 client.15150 trim_cache size 0 max 16384 2024-05-27T08:38:04.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.141+0000 7f12038b0b80 1 Processor -- start 2024-05-27T08:38:04.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.141+0000 7f12038b0b80 1 -- start start 2024-05-27T08:38:04.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 0x559df7666220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766b850 0x559df7670130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 0x559df7670670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f12038b0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559df7684930 con 0x559df766b850 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f12038b0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559df7685000 con 0x559df766c210 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f12038b0b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559df76856d0 con 0x559df75971f0 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766b850 0x559df7670130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766b850 0x559df7670130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33688/0 (socket says 172.21.15.77:33688) 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 -- 172.21.15.77:0/890919240 learned_addr learned my addr 172.21.15.77:0/890919240 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 0x559df7666220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f359e700 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 0x559df7670670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 -- 172.21.15.77:0/890919240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 msgr2=0x559df7666220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:04.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 0x559df7666220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 -- 172.21.15.77:0/890919240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 msgr2=0x559df7670670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 0x559df7670670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 -- 172.21.15.77:0/890919240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559df7677950 con 0x559df766b850 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11f259c700 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766b850 0x559df7670130 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009990 tx=0x7f11e8005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7296863229d861 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11d8ff9700 1 -- 172.21.15.77:0/890919240 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8027070 con 0x559df766b850 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11d8ff9700 1 -- 172.21.15.77:0/890919240 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e8003860 con 0x559df766b850 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.142+0000 7f11d8ff9700 1 -- 172.21.15.77:0/890919240 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e80055d0 con 0x559df766b850 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 -- 172.21.15.77:0/890919240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766b850 msgr2=0x559df7670130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766b850 0x559df7670130 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009990 tx=0x7f11e8005960 comp rx=0 tx=0).stop 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 -- 172.21.15.77:0/890919240 shutdown_connections 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766c210 0x559df7670670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 0x559df7666220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 --2- 172.21.15.77:0/890919240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766b850 0x559df7670130 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 -- 172.21.15.77:0/890919240 >> 172.21.15.77:0/890919240 conn(0x559df7662a80 msgr2=0x559df7673900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 -- 172.21.15.77:0/890919240 shutdown_connections 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 -- 172.21.15.77:0/890919240 wait complete. 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 -- start start 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.143+0000 7f12038b0b80 1 Processor -- start 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 0x559df768fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df7690190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f12038b0b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766c210 0x559df768aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f12038b0b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559df7686470 con 0x559df766c210 2024-05-27T08:38:04.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f12038b0b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x559df7685da0 con 0x559df766b850 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f12038b0b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559df7686b40 con 0x559df75971f0 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f12038b0b80 10 client.0 _ll_register_callbacks cb 0x559df7554900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766c210 0x559df768aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766c210 0x559df768aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33700/0 (socket says 172.21.15.77:33700) 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 -- 172.21.15.77:0/2411920863 learned_addr learned my addr 172.21.15.77:0/2411920863 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 0x559df768fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f259c700 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df7690190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 -- 172.21.15.77:0/2411920863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 msgr2=0x559df768fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 0x559df768fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 -- 172.21.15.77:0/2411920863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 msgr2=0x559df7690190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:04.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df7690190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 -- 172.21.15.77:0/2411920863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559df768b290 con 0x559df766c210 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f259c700 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df7690190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11f359e700 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766c210 0x559df768aa20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f11ec000f80 tx=0x7f11ec0b1580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20d1d830de84ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11d6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11d6ff5700 1 -- 172.21.15.77:0/2411920863 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ec0c9e60 con 0x559df766c210 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f11d6ff5700 1 -- 172.21.15.77:0/2411920863 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f11ec0a8070 con 0x559df766c210 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.144+0000 7f12038b0b80 1 -- 172.21.15.77:0/2411920863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11e8003860 con 0x559df766c210 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f11d6ff5700 1 -- 172.21.15.77:0/2411920863 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f11ec0a8980 con 0x559df766c210 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f12038b0b80 10 client.15156 fetch_fsmap learned FSMap version 28 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f12038b0b80 1 -- 172.21.15.77:0/2411920863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559df76f14f0 con 0x559df766c210 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f11d6ff5700 1 -- 172.21.15.77:0/2411920863 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e8003860 con 0x559df766c210 2024-05-27T08:38:04.153 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f11d6ff5700 1 -- 172.21.15.77:0/2411920863 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f11ec0a4520 con 0x559df766c210 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f12038b0b80 10 client.15156 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f12038b0b80 10 client.15156 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f12038b0b80 20 client.15156 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f12038b0b80 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x559df76712d0 0x559df76f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f12038b0b80 4 client.15156 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f12038b0b80 1 -- 172.21.15.77:0/2411920863 --> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] -- command(tid 0: {"prefix": "status"}) -- 0x559df75c5450 con 0x559df76712d0 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.145+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x559df76712d0 0x559df76f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f11f2d9d700 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x559df76712d0 0x559df76f4ba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3afebc75683b19d server_cookie=6cb07e53bbaaf65e in_seq=0 out_seq=0 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f11d6ff5700 10 client.15156 ms_handle_connect on v2:172.21.15.179:6838/2672999131 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f11d6ff5700 1 -- 172.21.15.77:0/2411920863 <== mds.1 v2:172.21.15.179:6838/2672999131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x559df75c5450 con 0x559df76712d0 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f11d6ff5700 10 client.15156 handle_command_reply: tid=0 2024-05-27T08:38:04.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 client.15156 shutdown 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 -- 172.21.15.77:0/2411920863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766c210 msgr2=0x559df768aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766c210 0x559df768aa20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f11ec000f80 tx=0x7f11ec0b1580 comp rx=0 tx=0).stop 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 -- 172.21.15.77:0/2411920863 shutdown_connections 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x559df76712d0 0x559df76f4ba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f11d6ff5700 0 client.15156 ms_handle_reset on v2:172.21.15.179:6838/2672999131 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559df766c210 0x559df768aa20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559df75971f0 0x559df768fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 --2- 172.21.15.77:0/2411920863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x559df766b850 0x559df7690190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 -- 172.21.15.77:0/2411920863 >> 172.21.15.77:0/2411920863 conn(0x559df7662a80 msgr2=0x559df766f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 -- 172.21.15.77:0/2411920863 shutdown_connections 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 1 -- 172.21.15.77:0/2411920863 wait complete. 2024-05-27T08:38:04.155 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.146+0000 7f12038b0b80 20 client.15156 trim_cache size 0 max 16384 2024-05-27T08:38:04.156 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:38:04.156 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:38:04.156 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-05-27T08:38:04.156 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24433, 2024-05-27T08:38:04.156 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:38:04.156 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:38:04.156 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:38:04.156 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 163.915109927, 2024-05-27T08:38:04.157 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:38:04.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:38:04.157 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:38:04.157 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 200.731895237 2024-05-27T08:38:04.157 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 -- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f11dc0b2880 msgr2=0x7f11dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f11dc0b2880 0x7f11dc0b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11e40099d0 tx=0x7f11e4008040 comp rx=0 tx=0).stop 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 -- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f41084b0 msgr2=0x7f11f4199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f41084b0 0x7f11f4199510 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00b970 tx=0x7f11ec00bc80 comp rx=0 tx=0).stop 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 -- 172.21.15.77:0/1507290225 shutdown_connections 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11f41084b0 0x7f11f4199510 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f11f4102470 0x7f11f4198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f11dc0b2880 0x7f11dc0b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.158 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 --2- 172.21.15.77:0/1507290225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f4102e30 0x7f11f4198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:04.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 -- 172.21.15.77:0/1507290225 >> 172.21.15.77:0/1507290225 conn(0x7f11f4079410 msgr2=0x7f11f4100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:04.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 -- 172.21.15.77:0/1507290225 shutdown_connections 2024-05-27T08:38:04.159 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:04.147+0000 7f11f901a700 1 -- 172.21.15.77:0/1507290225 wait complete. 2024-05-27T08:38:05.012 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:04 smithi077 ceph-mon[32007]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:38:05.014 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T08:38:05.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:04 smithi179 ceph-mon[36481]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:38:05.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:04 smithi092 ceph-mon[36049]: pgmap v378: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-05-27T08:38:05.254 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:06.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 -- 172.21.15.77:0/47041697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 msgr2=0x7fe264109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/47041697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 0x7fe264109990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe258009990 tx=0x7fe258009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 -- 172.21.15.77:0/47041697 shutdown_connections 2024-05-27T08:38:06.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/47041697 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe26406c650 0x7fe2641103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/47041697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 0x7fe264109990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.911 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/47041697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26406bc90 0x7fe26406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 -- 172.21.15.77:0/47041697 >> 172.21.15.77:0/47041697 conn(0x7fe264079410 msgr2=0x7fe264079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 -- 172.21.15.77:0/47041697 shutdown_connections 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 -- 172.21.15.77:0/47041697 wait complete. 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 Processor -- start 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.908+0000 7fe26ae7e700 1 -- start start 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26ae7e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe26406bc90 0x7fe26419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26ae7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26406c650 0x7fe26419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26ae7e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 0x7fe264199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26ae7e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe264113740 con 0x7fe2641095b0 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26ae7e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641135c0 con 0x7fe26406bc90 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26ae7e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe2641138c0 con 0x7fe26406c650 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 0x7fe264199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 0x7fe264199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33716/0 (socket says 172.21.15.77:33716) 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 -- 172.21.15.77:0/2281127810 learned_addr learned my addr 172.21.15.77:0/2281127810 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe263fff700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26406c650 0x7fe26419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe268c1a700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe26406bc90 0x7fe26419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 -- 172.21.15.77:0/2281127810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26406c650 msgr2=0x7fe26419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26406c650 0x7fe26419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 -- 172.21.15.77:0/2281127810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe26406bc90 msgr2=0x7fe26419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe26406bc90 0x7fe26419f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 -- 172.21.15.77:0/2281127810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26419a240 con 0x7fe2641095b0 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe268c1a700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe26406bc90 0x7fe26419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26941b700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 0x7fe264199ae0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00ada0 tx=0x7fe25c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe261ffb700 1 -- 172.21.15.77:0/2281127810 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c017020 con 0x7fe2641095b0 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe261ffb700 1 -- 172.21.15.77:0/2281127810 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25c00ce30 con 0x7fe2641095b0 2024-05-27T08:38:06.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe261ffb700 1 -- 172.21.15.77:0/2281127810 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0054a0 con 0x7fe2641095b0 2024-05-27T08:38:06.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.909+0000 7fe26ae7e700 1 -- 172.21.15.77:0/2281127810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26419a530 con 0x7fe2641095b0 2024-05-27T08:38:06.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.910+0000 7fe26ae7e700 1 -- 172.21.15.77:0/2281127810 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2641a67f0 con 0x7fe2641095b0 2024-05-27T08:38:06.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.912+0000 7fe261ffb700 1 -- 172.21.15.77:0/2281127810 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe25c00c980 con 0x7fe2641095b0 2024-05-27T08:38:06.916 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.912+0000 7fe261ffb700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe2540b2900 0x7fe2540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe268c1a700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe2540b2900 0x7fe2540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe261ffb700 1 -- 172.21.15.77:0/2281127810 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe25c0e1950 con 0x7fe2641095b0 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe268c1a700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe2540b2900 0x7fe2540b4dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b5c0 tx=0x7fe258005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe275714b80 1 Processor -- start 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe275714b80 1 -- start start 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe275714b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f126b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe275714b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f0ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe275714b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0d210 0x5640a3f19030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe275714b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5640a3e6ad20 con 0x5640a3f122d0 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe275714b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5640a3e5a8c0 con 0x5640a3f0d210 2024-05-27T08:38:06.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.913+0000 7fe275714b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640a3e5e900 con 0x5640a3f0c850 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f126b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f126b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33730/0 (socket says 172.21.15.77:33730) 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 -- 172.21.15.77:0/2516606763 learned_addr learned my addr 172.21.15.77:0/2516606763 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe26941b700 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0d210 0x5640a3f19030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe263fff700 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f0ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 -- 172.21.15.77:0/2516606763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 msgr2=0x5640a3f0ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f0ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 -- 172.21.15.77:0/2516606763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0d210 msgr2=0x5640a3f19030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0d210 0x5640a3f19030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 -- 172.21.15.77:0/2516606763 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a3e03f60 con 0x5640a3f122d0 2024-05-27T08:38:06.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe26941b700 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0d210 0x5640a3f19030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe268c1a700 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f126b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe25800ba90 tx=0x7fe25801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd54335c10a6fdd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe24effd700 1 -- 172.21.15.77:0/2516606763 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258003860 con 0x5640a3f122d0 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe24effd700 1 -- 172.21.15.77:0/2516606763 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe258005030 con 0x5640a3f122d0 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.914+0000 7fe24effd700 1 -- 172.21.15.77:0/2516606763 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25800f6d0 con 0x5640a3f122d0 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 -- 172.21.15.77:0/2516606763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 msgr2=0x5640a3f126b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f126b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe25800ba90 tx=0x7fe25801e000 comp rx=0 tx=0).stop 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 -- 172.21.15.77:0/2516606763 shutdown_connections 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0d210 0x5640a3f19030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f126b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 --2- 172.21.15.77:0/2516606763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f0ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 -- 172.21.15.77:0/2516606763 >> 172.21.15.77:0/2516606763 conn(0x5640a3e381f0 msgr2=0x5640a3f07390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:06.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 -- 172.21.15.77:0/2516606763 shutdown_connections 2024-05-27T08:38:06.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 -- 172.21.15.77:0/2516606763 wait complete. 2024-05-27T08:38:06.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 -- start start 2024-05-27T08:38:06.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.915+0000 7fe275714b80 1 Processor -- start 2024-05-27T08:38:06.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe275714b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 0x5640a3f18c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe275714b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 0x5640a3f15f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe275714b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe275714b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5640a3d60f50 con 0x5640a3f122d0 2024-05-27T08:38:06.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe275714b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5640a3dc2d50 con 0x5640a3f0c850 2024-05-27T08:38:06.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe275714b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640a3f19630 con 0x5640a3f0d210 2024-05-27T08:38:06.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe275714b80 10 client.0 _ll_register_callbacks cb 0x5640a3df5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f164a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33738/0 (socket says 172.21.15.77:33738) 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 -- 172.21.15.77:0/1038531648 learned_addr learned my addr 172.21.15.77:0/1038531648 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe268c1a700 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 0x5640a3f18c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe263fff700 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 0x5640a3f15f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 -- 172.21.15.77:0/1038531648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 msgr2=0x5640a3f15f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 0x5640a3f15f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 -- 172.21.15.77:0/1038531648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 msgr2=0x5640a3f18c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 0x5640a3f18c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 -- 172.21.15.77:0/1038531648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a3daa880 con 0x5640a3f122d0 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe26941b700 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f164a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe25c004fd0 tx=0x7fe25c00cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90055779dd51168a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.916+0000 7fe243fff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe243fff700 1 -- 172.21.15.77:0/1038531648 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0a9700 con 0x5640a3f122d0 2024-05-27T08:38:06.924 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe243fff700 1 -- 172.21.15.77:0/1038531648 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe25c0b1940 con 0x5640a3f122d0 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 1 -- 172.21.15.77:0/1038531648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe258005030 con 0x5640a3f122d0 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe243fff700 1 -- 172.21.15.77:0/1038531648 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe25c0a9000 con 0x5640a3f122d0 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 10 client.15168 fetch_fsmap learned FSMap version 28 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 1 -- 172.21.15.77:0/1038531648 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640a3e14710 con 0x5640a3f122d0 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe243fff700 1 -- 172.21.15.77:0/1038531648 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe258005030 con 0x5640a3f122d0 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe243fff700 1 -- 172.21.15.77:0/1038531648 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe25c012070 con 0x5640a3f122d0 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 10 client.15168 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 10 client.15168 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 20 client.15168 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5640a3f26130 0x5640a3f275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 4 client.15168 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe275714b80 1 -- 172.21.15.77:0/1038531648 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640a3e6cc70 con 0x5640a3f26130 2024-05-27T08:38:06.925 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.917+0000 7fe268c1a700 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5640a3f26130 0x5640a3f275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.918+0000 7fe268c1a700 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5640a3f26130 0x5640a3f275a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a82b4338c0ca74 server_cookie=d6429aad846aeaeb in_seq=0 out_seq=0 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.918+0000 7fe243fff700 10 client.15168 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.918+0000 7fe243fff700 1 -- 172.21.15.77:0/1038531648 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5640a3e6cc70 con 0x5640a3f26130 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.918+0000 7fe243fff700 10 client.15168 handle_command_reply: tid=0 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 client.15168 shutdown 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 -- 172.21.15.77:0/1038531648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 msgr2=0x5640a3f164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f164a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe25c004fd0 tx=0x7fe25c00cd40 comp rx=0 tx=0).stop 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 -- 172.21.15.77:0/1038531648 shutdown_connections 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5640a3f26130 0x5640a3f275a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f164a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 0x5640a3f18c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.926 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe243fff700 0 client.15168 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:38:06.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 --2- 172.21.15.77:0/1038531648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 0x5640a3f15f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 -- 172.21.15.77:0/1038531648 >> 172.21.15.77:0/1038531648 conn(0x5640a3e381f0 msgr2=0x5640a3f06aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:06.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 -- 172.21.15.77:0/1038531648 shutdown_connections 2024-05-27T08:38:06.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 1 -- 172.21.15.77:0/1038531648 wait complete. 2024-05-27T08:38:06.927 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.919+0000 7fe275714b80 20 client.15168 trim_cache size 0 max 16384 2024-05-27T08:38:06.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe275714b80 1 Processor -- start 2024-05-27T08:38:06.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe275714b80 1 -- start start 2024-05-27T08:38:06.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe275714b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 0x5640a3f11920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe275714b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 0x5640a3f17ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe275714b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f14740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe275714b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5640a3f258e0 con 0x5640a3f122d0 2024-05-27T08:38:06.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe275714b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5640a3f25fb0 con 0x5640a3f0c850 2024-05-27T08:38:06.942 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe275714b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640a3f26680 con 0x5640a3f0d210 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f14740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f14740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33750/0 (socket says 172.21.15.77:33750) 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 -- 172.21.15.77:0/1495359302 learned_addr learned my addr 172.21.15.77:0/1495359302 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe263fff700 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 0x5640a3f17ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe268c1a700 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 0x5640a3f11920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 -- 172.21.15.77:0/1495359302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 msgr2=0x5640a3f17ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 0x5640a3f17ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 -- 172.21.15.77:0/1495359302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 msgr2=0x5640a3f11920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 0x5640a3f11920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.943 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 -- 172.21.15.77:0/1495359302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a3f23ba0 con 0x5640a3f122d0 2024-05-27T08:38:06.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.940+0000 7fe26941b700 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f14740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe25c004dc0 tx=0x7fe25c0a8bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77271a9673ea7578 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:06.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe24e7fc700 1 -- 172.21.15.77:0/1495359302 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0ad3b0 con 0x5640a3f122d0 2024-05-27T08:38:06.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe24e7fc700 1 -- 172.21.15.77:0/1495359302 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25c0b1dd0 con 0x5640a3f122d0 2024-05-27T08:38:06.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe24e7fc700 1 -- 172.21.15.77:0/1495359302 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0a8010 con 0x5640a3f122d0 2024-05-27T08:38:06.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 -- 172.21.15.77:0/1495359302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 msgr2=0x5640a3f14740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f14740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe25c004dc0 tx=0x7fe25c0a8bd0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 -- 172.21.15.77:0/1495359302 shutdown_connections 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f122d0 0x5640a3f14740 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f0c850 0x5640a3f11920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 --2- 172.21.15.77:0/1495359302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0d210 0x5640a3f17ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 -- 172.21.15.77:0/1495359302 >> 172.21.15.77:0/1495359302 conn(0x5640a3e381f0 msgr2=0x5640a3f17790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 -- 172.21.15.77:0/1495359302 shutdown_connections 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.941+0000 7fe275714b80 1 -- 172.21.15.77:0/1495359302 wait complete. 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 1 -- start start 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 1 Processor -- start 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f30e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f0d210 0x5640a3f2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f122d0 0x5640a3f2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5640a3f27420 con 0x5640a3f0d210 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5640a3f26d50 con 0x5640a3f122d0 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640a3f27af0 con 0x5640a3f0c850 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe275714b80 10 client.0 _ll_register_callbacks cb 0x5640a3df5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe263fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f0d210 0x5640a3f2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe263fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f0d210 0x5640a3f2ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33756/0 (socket says 172.21.15.77:33756) 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe263fff700 1 -- 172.21.15.77:0/147158955 learned_addr learned my addr 172.21.15.77:0/147158955 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe26941b700 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f122d0 0x5640a3f2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe268c1a700 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f30e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.942+0000 7fe263fff700 1 -- 172.21.15.77:0/147158955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 msgr2=0x5640a3f30e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe263fff700 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f30e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe263fff700 1 -- 172.21.15.77:0/147158955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f122d0 msgr2=0x5640a3f2e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe263fff700 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f122d0 0x5640a3f2e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe263fff700 1 -- 172.21.15.77:0/147158955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a3f31440 con 0x5640a3f0d210 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe268c1a700 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f30e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe263fff700 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f0d210 0x5640a3f2ddd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe264079a60 tx=0x7fe250009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebecb99455937b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe243fff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe243fff700 1 -- 172.21.15.77:0/147158955 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25001a030 con 0x5640a3f0d210 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe243fff700 1 -- 172.21.15.77:0/147158955 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fe250013ad0 con 0x5640a3f0d210 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 1 -- 172.21.15.77:0/147158955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe25c0b1dd0 con 0x5640a3f0d210 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe243fff700 1 -- 172.21.15.77:0/147158955 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe25000f040 con 0x5640a3f0d210 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 10 client.15174 fetch_fsmap learned FSMap version 28 2024-05-27T08:38:06.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 1 -- 172.21.15.77:0/147158955 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5640a3f963e0 con 0x5640a3f0d210 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe243fff700 1 -- 172.21.15.77:0/147158955 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe25c0b1dd0 con 0x5640a3f0d210 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe243fff700 1 -- 172.21.15.77:0/147158955 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe25001f410 con 0x5640a3f0d210 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 28 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 10 client.15174 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 20 client.15174 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5640a3f9b090 0x5640a3f99da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 4 client.15174 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.943+0000 7fe275714b80 1 -- 172.21.15.77:0/147158955 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "status"}) -- 0x5640a3e66450 con 0x5640a3f9b090 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.944+0000 7fe268c1a700 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5640a3f9b090 0x5640a3f99da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.944+0000 7fe268c1a700 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5640a3f9b090 0x5640a3f99da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6270fc88460e114 server_cookie=85e3f7df5bbf1857 in_seq=0 out_seq=0 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.944+0000 7fe243fff700 10 client.15174 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.944+0000 7fe243fff700 1 -- 172.21.15.77:0/147158955 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5640a3e66450 con 0x5640a3f9b090 2024-05-27T08:38:06.951 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.944+0000 7fe243fff700 10 client.15174 handle_command_reply: tid=0 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.944+0000 7fe275714b80 1 client.15174 shutdown 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 -- 172.21.15.77:0/147158955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f0d210 msgr2=0x5640a3f2ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f0d210 0x5640a3f2ddd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe264079a60 tx=0x7fe250009ef0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 -- 172.21.15.77:0/147158955 shutdown_connections 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5640a3f9b090 0x5640a3f99da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5640a3f122d0 0x5640a3f2e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe243fff700 0 client.15174 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a3f0c850 0x5640a3f30e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 --2- 172.21.15.77:0/147158955 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640a3f0d210 0x5640a3f2ddd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 -- 172.21.15.77:0/147158955 >> 172.21.15.77:0/147158955 conn(0x5640a3e381f0 msgr2=0x5640a3f117a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 -- 172.21.15.77:0/147158955 shutdown_connections 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 1 -- 172.21.15.77:0/147158955 wait complete. 2024-05-27T08:38:06.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.945+0000 7fe275714b80 20 client.15174 trim_cache size 0 max 16384 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24434, 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 177.90373946, 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 28, 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 85, 2024-05-27T08:38:06.953 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 83, 2024-05-27T08:38:06.954 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 193.29072126299999 2024-05-27T08:38:06.954 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:38:06.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 -- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe2540b2900 msgr2=0x7fe2540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe2540b2900 0x7fe2540b4dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b5c0 tx=0x7fe258005fb0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 -- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 msgr2=0x7fe264199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 0x7fe264199ae0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00ada0 tx=0x7fe25c00c330 comp rx=0 tx=0).stop 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 -- 172.21.15.77:0/2281127810 shutdown_connections 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2641095b0 0x7fe264199ae0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe26406bc90 0x7fe26419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fe2540b2900 0x7fe2540b4dc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 --2- 172.21.15.77:0/2281127810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe26406c650 0x7fe26419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 -- 172.21.15.77:0/2281127810 >> 172.21.15.77:0/2281127810 conn(0x7fe264079410 msgr2=0x7fe2641011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 -- 172.21.15.77:0/2281127810 shutdown_connections 2024-05-27T08:38:06.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:06.946+0000 7fe26ae7e700 1 -- 172.21.15.77:0/2281127810 wait complete. 2024-05-27T08:38:06.958 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:06 smithi077 ceph-mon[32007]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:38:07.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:06 smithi179 ceph-mon[36481]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:38:07.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:06 smithi092 ceph-mon[36049]: pgmap v379: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-05-27T08:38:07.740 INFO:teuthology.run_tasks:Running task sequential... 2024-05-27T08:38:07.750 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-27T08:38:07.750 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-27T08:38:07.751 INFO:teuthology.task.print:Enabling mgr modules 2024-05-27T08:38:07.751 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-27T08:38:07.752 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T08:38:07.752 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:38:07.752 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 -- 172.21.15.77:0/766840380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8798102bb0 msgr2=0x7f8798103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 --2- 172.21.15.77:0/766840380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8798102bb0 0x7f8798103030 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802a410 comp rx=0 tx=0).stop 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 -- 172.21.15.77:0/766840380 shutdown_connections 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 --2- 172.21.15.77:0/766840380 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8798103570 0x7f8798109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 --2- 172.21.15.77:0/766840380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8798102bb0 0x7f8798103030 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 --2- 172.21.15.77:0/766840380 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f87981019c0 0x7f8798101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 -- 172.21.15.77:0/766840380 >> 172.21.15.77:0/766840380 conn(0x7f87980fcf10 msgr2=0x7f87980ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 -- 172.21.15.77:0/766840380 shutdown_connections 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.855+0000 7f87a0589640 1 -- 172.21.15.77:0/766840380 wait complete. 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f87a0589640 1 Processor -- start 2024-05-27T08:38:07.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f87a0589640 1 -- start start 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f87a0589640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87981019c0 0x7f879806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f87a0589640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8798102bb0 0x7f879819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f87a0589640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798103570 0x7f87981a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f87a0589640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f879810d5e0 con 0x7f87981019c0 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f87a0589640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f879810d460 con 0x7f8798103570 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f87a0589640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f879810d760 con 0x7f8798102bb0 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879e325640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87981019c0 0x7f879806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879e325640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87981019c0 0x7f879806beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33774/0 (socket says 172.21.15.77:33774) 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879e325640 1 -- 172.21.15.77:0/3487588828 learned_addr learned my addr 172.21.15.77:0/3487588828 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879db24640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8798102bb0 0x7f879819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879e325640 1 -- 172.21.15.77:0/3487588828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8798102bb0 msgr2=0x7f879819da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879e325640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8798102bb0 0x7f879819da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879e325640 1 -- 172.21.15.77:0/3487588828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798103570 msgr2=0x7f87981a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:07.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879e325640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798103570 0x7f87981a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:07.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.856+0000 7f879e325640 1 -- 172.21.15.77:0/3487588828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87981a09b0 con 0x7f87981019c0 2024-05-27T08:38:07.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.857+0000 7f879e325640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87981019c0 0x7f879806beb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f879400b450 tx=0x7f879400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:07.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.857+0000 7f878f7fe640 1 -- 172.21.15.77:0/3487588828 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8794015020 con 0x7f87981019c0 2024-05-27T08:38:07.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.857+0000 7f878f7fe640 1 -- 172.21.15.77:0/3487588828 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8794011d40 con 0x7f87981019c0 2024-05-27T08:38:07.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.857+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87981a0ca0 con 0x7f87981019c0 2024-05-27T08:38:07.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.857+0000 7f878f7fe640 1 -- 172.21.15.77:0/3487588828 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879400f340 con 0x7f87981019c0 2024-05-27T08:38:07.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.857+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87981a11a0 con 0x7f87981019c0 2024-05-27T08:38:07.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.858+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8798108ce0 con 0x7f87981019c0 2024-05-27T08:38:07.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.859+0000 7f878f7fe640 1 -- 172.21.15.77:0/3487588828 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8794018020 con 0x7f87981019c0 2024-05-27T08:38:07.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.860+0000 7f878f7fe640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f87840b28a0 0x7f87840b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:07.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.860+0000 7f879db24640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f87840b28a0 0x7f87840b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:07.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.860+0000 7f878f7fe640 1 -- 172.21.15.77:0/3487588828 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f87940e2310 con 0x7f87981019c0 2024-05-27T08:38:07.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.861+0000 7f879db24640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f87840b28a0 0x7f87840b4d60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f879806bc20 tx=0x7f87880314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:07.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:07.864+0000 7f878f7fe640 1 -- 172.21.15.77:0/3487588828 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f879401f020 con 0x7f87981019c0 2024-05-27T08:38:08.098 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.097+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f879802de10 con 0x7f87981019c0 2024-05-27T08:38:08.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.775+0000 7f878f7fe640 1 -- 172.21.15.77:0/3487588828 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f879400d040 con 0x7f87981019c0 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f87840b28a0 msgr2=0x7f87840b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f87840b28a0 0x7f87840b4d60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f879806bc20 tx=0x7f87880314a0 comp rx=0 tx=0).stop 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87981019c0 msgr2=0x7f879806beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87981019c0 0x7f879806beb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f879400b450 tx=0x7f879400b910 comp rx=0 tx=0).stop 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 shutdown_connections 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f87840b28a0 0x7f87840b4d60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8798102bb0 0x7f879819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8798103570 0x7f87981a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 --2- 172.21.15.77:0/3487588828 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87981019c0 0x7f879806beb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 >> 172.21.15.77:0/3487588828 conn(0x7f87980fcf10 msgr2=0x7f8798105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:08.780 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.778+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 shutdown_connections 2024-05-27T08:38:08.781 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:08.779+0000 7f87a0589640 1 -- 172.21.15.77:0/3487588828 wait complete. 2024-05-27T08:38:08.799 INFO:teuthology.run_tasks:Running task kclient... 2024-05-27T08:38:08.828 INFO:tasks.kclient:Mounting kernel clients... 2024-05-27T08:38:08.829 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-27T08:38:08.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:08.829 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-05-27T08:38:08.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:08.848 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2024-05-27T08:38:08.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:38:08.918 INFO:teuthology.orchestra.run.smithi077.stderr:Cannot find device "ceph-brx" 2024-05-27T08:38:08.918 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-27T08:38:08.918 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-27T08:38:08.919 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T08:38:08.919 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-05-27T08:38:09.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:08 smithi179 ceph-mon[36481]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:09.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:08 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3487588828' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T08:38:09.107 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: ignoring --setuser ceph since I am not root 2024-05-27T08:38:09.107 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: ignoring --setgroup ceph since I am not root 2024-05-27T08:38:09.107 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:08 smithi179 ceph-mgr[38326]: -- 172.21.15.179:0/1242169717 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b036405a0 con 0x559b03615000 2024-05-27T08:38:09.107 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:08.957+0000 7f853e769200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T08:38:09.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:08 smithi077 ceph-mon[32007]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:09.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:08 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3487588828' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T08:38:09.130 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: ignoring --setuser ceph since I am not root 2024-05-27T08:38:09.130 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: ignoring --setgroup ceph since I am not root 2024-05-27T08:38:09.130 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:08.967+0000 7f1daa106200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T08:38:09.140 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:08 smithi092 ceph-mon[36049]: pgmap v380: 97 pgs: 97 active+clean; 643 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:09.140 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:08 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3487588828' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-27T08:38:09.141 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:08 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: ignoring --setuser ceph since I am not root 2024-05-27T08:38:09.141 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:08 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: ignoring --setgroup ceph since I am not root 2024-05-27T08:38:09.141 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:08 smithi092 ceph-mgr[37427]: -- 172.21.15.92:0/4186970277 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f6321e5a0 con 0x556f631f2800 2024-05-27T08:38:09.141 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:08 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:08.954+0000 7ff4526bc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-27T08:38:09.141 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:09 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:09.138+0000 7ff4526bc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:38:09.209 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:09.466 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:09.166+0000 7f1daa106200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:38:09.466 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:09.281+0000 7f1daa106200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T08:38:09.486 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:09 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:09.241+0000 7ff4526bc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T08:38:09.555 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:09.141+0000 7f853e769200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:38:09.555 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:09.242+0000 7f853e769200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-27T08:38:09.556 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:09.554+0000 7f853e769200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T08:38:09.796 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:09.605+0000 7f1daa106200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T08:38:09.796 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:09.693+0000 7f1daa106200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T08:38:09.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:09 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3487588828' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T08:38:09.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:09 smithi179 ceph-mon[36481]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-05-27T08:38:09.856 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:09.640+0000 7f853e769200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T08:38:09.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:09 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3487588828' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T08:38:09.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:09 smithi092 ceph-mon[36049]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-05-27T08:38:09.986 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:09 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:09.552+0000 7ff4526bc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-27T08:38:09.986 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:09 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:09.638+0000 7ff4526bc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-27T08:38:10.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:09 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3487588828' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-27T08:38:10.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:09 smithi077 ceph-mon[32007]: mgrmap e19: x(active, since 10m), standbys: y, z 2024-05-27T08:38:10.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f79513ea700 1 -- 172.21.15.77:0/1984292735 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948003b40 con 0x7f794c109520 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 -- 172.21.15.77:0/1984292735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c109520 msgr2=0x7f794c109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 --2- 172.21.15.77:0/1984292735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c109520 0x7f794c109900 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f79480099c0 tx=0x7f7948009cd0 comp rx=0 tx=0).stop 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 -- 172.21.15.77:0/1984292735 shutdown_connections 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 --2- 172.21.15.77:0/1984292735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f794c1024f0 0x7f794c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 --2- 172.21.15.77:0/1984292735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c109520 0x7f794c109900 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 --2- 172.21.15.77:0/1984292735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f794c101b30 0x7f794c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 -- 172.21.15.77:0/1984292735 >> 172.21.15.77:0/1984292735 conn(0x7f794c078ab0 msgr2=0x7f794c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 -- 172.21.15.77:0/1984292735 shutdown_connections 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.893+0000 7f7954650700 1 -- 172.21.15.77:0/1984292735 wait complete. 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7954650700 1 Processor -- start 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7954650700 1 -- start start 2024-05-27T08:38:10.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7954650700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f794c101b30 0x7f794c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7954650700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c1024f0 0x7f794c19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7954650700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f794c109520 0x7f794c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7954650700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f794c1137a0 con 0x7f794c1024f0 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7954650700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f794c113620 con 0x7f794c109520 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7954650700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f794c113920 con 0x7f794c101b30 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7951beb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c1024f0 0x7f794c19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7951beb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c1024f0 0x7f794c19c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48884/0 (socket says 172.21.15.77:48884) 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7951beb700 1 -- 172.21.15.77:0/3387554773 learned_addr learned my addr 172.21.15.77:0/3387554773 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f79523ec700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f794c101b30 0x7f794c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:10.897 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.894+0000 7f7952bed700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f794c109520 0x7f794c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:10.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f7951beb700 1 -- 172.21.15.77:0/3387554773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f794c101b30 msgr2=0x7f794c19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:10.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f7951beb700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f794c101b30 0x7f794c19bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:10.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f7951beb700 1 -- 172.21.15.77:0/3387554773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f794c109520 msgr2=0x7f794c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:10.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f7951beb700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f794c109520 0x7f794c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:10.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f7951beb700 1 -- 172.21.15.77:0/3387554773 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c1a0de0 con 0x7f794c1024f0 2024-05-27T08:38:10.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f7951beb700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c1024f0 0x7f794c19c310 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e9a0 tx=0x7f793c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:10.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f79437fe700 1 -- 172.21.15.77:0/3387554773 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c00ca10 con 0x7f794c1024f0 2024-05-27T08:38:10.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f79437fe700 1 -- 172.21.15.77:0/3387554773 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793c00cbb0 con 0x7f794c1024f0 2024-05-27T08:38:10.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794c1a10d0 con 0x7f794c1024f0 2024-05-27T08:38:10.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f79437fe700 1 -- 172.21.15.77:0/3387554773 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c018680 con 0x7f794c1024f0 2024-05-27T08:38:10.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.895+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794c1a8950 con 0x7f794c1024f0 2024-05-27T08:38:10.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.896+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794c06a230 con 0x7f794c1024f0 2024-05-27T08:38:10.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.898+0000 7f79437fe700 1 -- 172.21.15.77:0/3387554773 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f793c010430 con 0x7f794c1024f0 2024-05-27T08:38:10.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.898+0000 7f79437fe700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f79380b28d0 0x7f79380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:10.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.898+0000 7f79523ec700 1 -- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f79380b28d0 msgr2=0x7f79380b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:10.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.898+0000 7f79437fe700 1 -- 172.21.15.77:0/3387554773 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f793c0e2720 con 0x7f794c1024f0 2024-05-27T08:38:10.900 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.898+0000 7f79523ec700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f79380b28d0 0x7f79380b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:38:10.907 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:10.906+0000 7f79437fe700 1 -- 172.21.15.77:0/3387554773 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f793c0aa440 con 0x7f794c1024f0 2024-05-27T08:38:11.093 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:10 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:10.915+0000 7f1daa106200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T08:38:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.099+0000 7f79523ec700 1 -- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f79380b28d0 msgr2=0x7f79380b4d90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:11.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.099+0000 7f79523ec700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f79380b28d0 0x7f79380b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:38:11.157 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:10.816+0000 7ff4526bc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T08:38:11.157 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:10.985+0000 7ff4526bc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T08:38:11.158 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:11.156+0000 7ff4526bc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T08:38:11.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.169+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f794c19d130 con 0x7f794c1024f0 2024-05-27T08:38:11.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.169+0000 7f79437fe700 1 -- 172.21.15.77:0/3387554773 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f794c19d130 con 0x7f794c1024f0 2024-05-27T08:38:11.171 INFO:teuthology.orchestra.run.smithi077.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-27T08:38:11.173 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f79380b28d0 msgr2=0x7f79380b4d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f79380b28d0 0x7f79380b4d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c1024f0 msgr2=0x7f794c19c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c1024f0 0x7f794c19c310 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e9a0 tx=0x7f793c00ecb0 comp rx=0 tx=0).stop 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 shutdown_connections 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f794c109520 0x7f794c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f79380b28d0 0x7f79380b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f794c101b30 0x7f794c19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 --2- 172.21.15.77:0/3387554773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f794c1024f0 0x7f794c19c310 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 >> 172.21.15.77:0/3387554773 conn(0x7f794c078ab0 msgr2=0x7f794c100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 shutdown_connections 2024-05-27T08:38:11.174 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11.172+0000 7f7954650700 1 -- 172.21.15.77:0/3387554773 wait complete. 2024-05-27T08:38:11.180 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:10.880+0000 7f853e769200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-27T08:38:11.180 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:11.058+0000 7f853e769200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T08:38:11.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:11 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3387554773' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T08:38:11.380 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:11.092+0000 7f1daa106200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-27T08:38:11.380 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:11.270+0000 7f1daa106200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T08:38:11.436 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:11 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3387554773' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T08:38:11.528 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:11 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3387554773' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-27T08:38:11.529 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:11.236+0000 7f853e769200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-27T08:38:11.529 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:11.527+0000 7f853e769200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T08:38:11.737 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:11.458+0000 7ff4526bc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T08:38:11.737 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:11.557+0000 7ff4526bc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T08:38:11.824 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:11.618+0000 7f853e769200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T08:38:11.824 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:11.823+0000 7f853e769200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T08:38:11.855 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:11.569+0000 7f1daa106200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-27T08:38:11.855 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:11.661+0000 7f1daa106200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-27T08:38:11.884 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-27T08:38:11.884 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-27T08:38:11.884 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi077.front.sepia.ceph.com 2024-05-27T08:38:11.884 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-27T08:38:11.884 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-27T08:38:11.884 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-27T08:38:11.884 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-27T08:38:11.884 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-27T08:38:11.885 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-27T08:38:11.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:11.885 DEBUG:teuthology.orchestra.run.smithi077:> ip addr 2024-05-27T08:38:11.901 INFO:teuthology.orchestra.run.smithi077.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-27T08:38:11.901 INFO:teuthology.orchestra.run.smithi077.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-27T08:38:11.901 INFO:teuthology.orchestra.run.smithi077.stdout: inet 127.0.0.1/8 scope host lo 2024-05-27T08:38:11.901 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-05-27T08:38:11.901 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 ::1/128 scope host 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d8 brd ff:ff:ff:ff:ff:ff 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f0 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:d8:d9:d9 brd ff:ff:ff:ff:ff:ff 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: altname enp7s0f1 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d2 brd ff:ff:ff:ff:ff:ff 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: link/ether 0c:c4:7a:8f:d3:d3 brd ff:ff:ff:ff:ff:ff 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: inet 172.21.15.77/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-27T08:38:11.902 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft 42057sec preferred_lft 42057sec 2024-05-27T08:38:11.903 INFO:teuthology.orchestra.run.smithi077.stdout: inet6 fe80::ec4:7aff:fe8f:d3d3/64 scope link 2024-05-27T08:38:11.903 INFO:teuthology.orchestra.run.smithi077.stdout: valid_lft forever preferred_lft forever 2024-05-27T08:38:11.903 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-27T08:38:11.903 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T08:38:11.903 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-05-27T08:38:11.903 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add name ceph-brx type bridge 2024-05-27T08:38:11.903 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr flush dev ceph-brx 2024-05-27T08:38:11.903 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set ceph-brx up 2024-05-27T08:38:11.903 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-27T08:38:11.904 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-05-27T08:38:11.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T08:38:12.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:12.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:12.103 DEBUG:teuthology.orchestra.run.smithi077:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-27T08:38:12.106 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:12.017+0000 7f853e769200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:38:12.122 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:11.871+0000 7f1daa106200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T08:38:12.122 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:12.082+0000 7f1daa106200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:38:12.127 INFO:teuthology.orchestra.run.smithi077.stdout:1 2024-05-27T08:38:12.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:12.129 DEBUG:teuthology.orchestra.run.smithi077:> route 2024-05-27T08:38:12.196 INFO:teuthology.orchestra.run.smithi077.stdout:Kernel IP routing table 2024-05-27T08:38:12.196 INFO:teuthology.orchestra.run.smithi077.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-27T08:38:12.196 INFO:teuthology.orchestra.run.smithi077.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-27T08:38:12.196 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-27T08:38:12.197 INFO:teuthology.orchestra.run.smithi077.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-27T08:38:12.197 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T08:38:12.197 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-05-27T08:38:12.197 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-27T08:38:12.197 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-27T08:38:12.197 DEBUG:teuthology.orchestra.run.smithi077:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-27T08:38:12.197 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-05-27T08:38:12.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:11.767+0000 7ff4526bc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-27T08:38:12.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:11.969+0000 7ff4526bc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-27T08:38:12.285 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T08:38:12.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:12.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:12.511 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-05-27T08:38:12.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:12.528 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list-id 2024-05-27T08:38:12.583 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T08:38:12.583 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-05-27T08:38:12.583 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T08:38:12.583 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-27T08:38:12.584 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-05-27T08:38:12.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T08:38:12.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:12.703 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-27T08:38:12.703 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T08:38:12.704 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-05-27T08:38:12.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-27T08:38:12.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-27T08:38:12.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-27T08:38:12.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-27T08:38:12.704 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-27T08:38:12.704 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-05-27T08:38:12.736 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:12.583+0000 7ff4526bc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T08:38:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T08:38:12.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:12.830 DEBUG:teuthology.orchestra.run.smithi077:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-27T08:38:12.830 DEBUG:teuthology.orchestra.run.smithi077:> set -e 2024-05-27T08:38:12.830 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set brx.0 up 2024-05-27T08:38:12.830 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link set dev brx.0 master ceph-brx 2024-05-27T08:38:12.830 DEBUG:teuthology.orchestra.run.smithi077:> ') 2024-05-27T08:38:12.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T08:38:12.902 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:12.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T08:38:12.909 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-27T08:38:12.924 INFO:teuthology.orchestra.run.smithi077.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-27T08:38:12.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-27T08:38:12.925 DEBUG:teuthology.orchestra.run.smithi077:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-27T08:38:12.971 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:12.725+0000 7f1daa106200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T08:38:12.971 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:12.970+0000 7f1daa106200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T08:38:12.981 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-27T08:38:12.981 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-27T08:38:12.981 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/a4abe728-ad7d-4a7c-a93a-78a659276dc3 2024-05-27T08:38:12.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:12.981 DEBUG:teuthology.orchestra.run.smithi077:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/a4abe728-ad7d-4a7c-a93a-78a659276dc3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-27T08:38:13.097 INFO:teuthology.orchestra.run.smithi077.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-27T08:38:13.097 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-27T08:38:13.106 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:12.622+0000 7f853e769200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-27T08:38:13.106 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:12.842+0000 7f853e769200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T08:38:13.227 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:12.803+0000 7ff4526bc200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-27T08:38:13.229 INFO:teuthology.orchestra.run.smithi077.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-05-27T08:38:13.229 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-05-27T08:38:13.229 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: resolved to: "172.21.15.77:3300,172.21.15.92:3300,172.21.15.179:3300" 2024-05-27T08:38:13.229 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: trying mount with new device syntax: 0@c4eeaaa6-1c02-11ef-bc9b-c7b262605968.cephfs=/volumes/qa/sv_1/a4abe728-ad7d-4a7c-a93a-78a659276dc3 2024-05-27T08:38:13.229 INFO:teuthology.orchestra.run.smithi077.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.77:3300/172.21.15.92:3300/172.21.15.179:3300" will pass to kernel 2024-05-27T08:38:13.230 INFO:teuthology.orchestra.run.smithi077.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-27T08:38:13.230 INFO:teuthology.orchestra.run.smithi077.stdout: You just mounted a file system that supports labels which does not 2024-05-27T08:38:13.230 INFO:teuthology.orchestra.run.smithi077.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-27T08:38:13.230 INFO:teuthology.orchestra.run.smithi077.stdout: applications will generate AVC messages and not be allowed access to 2024-05-27T08:38:13.231 INFO:teuthology.orchestra.run.smithi077.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-27T08:38:13.232 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-27T08:38:13.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:38:13.232 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-27T08:38:13.258 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T08:38:13.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T08:38:13.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:13.357 INFO:teuthology.orchestra.run.smithi077.stdout:client15207 2024-05-27T08:38:13.357 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:38:13.357 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:38:13.375 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T08:38:13.464 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T08:38:13.466 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:13.472 INFO:teuthology.orchestra.run.smithi077.stdout:client15207 2024-05-27T08:38:13.472 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-27T08:38:13.486 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:13.225+0000 7ff4526bc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T08:38:13.486 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:13.323+0000 7ff4526bc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T08:38:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-27T08:38:13.524 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:13.530 INFO:teuthology.orchestra.run.smithi077.stdout:client15207 2024-05-27T08:38:13.530 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968.client15207/status') 2024-05-27T08:38:13.533 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:13.426+0000 7f1daa106200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T08:38:13.533 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:13.531+0000 7f1daa106200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T08:38:13.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-27T08:38:13.590 INFO:teuthology.orchestra.run.smithi077.stdout:instance: client.15207 (3)192.168.0.1:0/2693526231 2024-05-27T08:38:13.590 INFO:teuthology.orchestra.run.smithi077.stdout:blocklisted: false 2024-05-27T08:38:13.590 INFO:teuthology.orchestra.run.smithi077.stderr:0+1 records in 2024-05-27T08:38:13.590 INFO:teuthology.orchestra.run.smithi077.stderr:0+1 records out 2024-05-27T08:38:13.591 INFO:teuthology.orchestra.run.smithi077.stderr:70 bytes copied, 2.5096e-05 s, 2.8 MB/s 2024-05-27T08:38:13.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-27T08:38:13.597 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T08:38:13.607 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T08:38:13.607 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:38:13.607 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-27T08:38:13.608 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:13.265+0000 7f853e769200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-27T08:38:13.608 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:13.362+0000 7f853e769200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-27T08:38:13.717 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 -- 172.21.15.77:0/2061959537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd184100af0 msgr2=0x7fd184100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 --2- 172.21.15.77:0/2061959537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd184100af0 0x7fd184100f70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009900 tx=0x7fd16c02a410 comp rx=0 tx=0).stop 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 -- 172.21.15.77:0/2061959537 shutdown_connections 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 --2- 172.21.15.77:0/2061959537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1841014b0 0x7fd18410b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 --2- 172.21.15.77:0/2061959537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd184100af0 0x7fd184100f70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 --2- 172.21.15.77:0/2061959537 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184104080 0x7fd1841044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 -- 172.21.15.77:0/2061959537 >> 172.21.15.77:0/2061959537 conn(0x7fd1840fc6b0 msgr2=0x7fd1840fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 -- 172.21.15.77:0/2061959537 shutdown_connections 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.714+0000 7fd1890c8640 1 -- 172.21.15.77:0/2061959537 wait complete. 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd1890c8640 1 Processor -- start 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd1890c8640 1 -- start start 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd1890c8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd184100af0 0x7fd184198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:13.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd1890c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1841014b0 0x7fd184199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd1890c8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184104080 0x7fd1841a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd1890c8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd18410e5e0 con 0x7fd1841014b0 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd1890c8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd18410e460 con 0x7fd184104080 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd1890c8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd18410e760 con 0x7fd184100af0 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd17bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1841014b0 0x7fd184199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd17bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1841014b0 0x7fd184199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48910/0 (socket says 172.21.15.77:48910) 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd17bfff640 1 -- 172.21.15.77:0/3403469127 learned_addr learned my addr 172.21.15.77:0/3403469127 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.715+0000 7fd182d9d640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd184100af0 0x7fd184198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd18359e640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184104080 0x7fd1841a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd17bfff640 1 -- 172.21.15.77:0/3403469127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd184100af0 msgr2=0x7fd184198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd17bfff640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd184100af0 0x7fd184198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd17bfff640 1 -- 172.21.15.77:0/3403469127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184104080 msgr2=0x7fd1841a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:13.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd17bfff640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184104080 0x7fd1841a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:13.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd17bfff640 1 -- 172.21.15.77:0/3403469127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1841a0830 con 0x7fd1841014b0 2024-05-27T08:38:13.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd17bfff640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1841014b0 0x7fd184199160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd16c005940 tx=0x7fd16c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:13.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd180d99640 1 -- 172.21.15.77:0/3403469127 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c03f030 con 0x7fd1841014b0 2024-05-27T08:38:13.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1841a0ac0 con 0x7fd1841014b0 2024-05-27T08:38:13.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd180d99640 1 -- 172.21.15.77:0/3403469127 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd16c003cb0 con 0x7fd1841014b0 2024-05-27T08:38:13.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd180d99640 1 -- 172.21.15.77:0/3403469127 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c035070 con 0x7fd1841014b0 2024-05-27T08:38:13.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.716+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1841a1000 con 0x7fd1841014b0 2024-05-27T08:38:13.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.717+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd184109f40 con 0x7fd1841014b0 2024-05-27T08:38:13.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.719+0000 7fd180d99640 1 -- 172.21.15.77:0/3403469127 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fd16c042030 con 0x7fd1841014b0 2024-05-27T08:38:13.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.719+0000 7fd180d99640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd1700b28f0 0x7fd1700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:13.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.720+0000 7fd182d9d640 1 -- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd1700b28f0 msgr2=0x7fd1700b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:13.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.720+0000 7fd180d99640 1 -- 172.21.15.77:0/3403469127 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7fd16c102d50 con 0x7fd1841014b0 2024-05-27T08:38:13.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.720+0000 7fd182d9d640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd1700b28f0 0x7fd1700b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:38:13.730 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.729+0000 7fd180d99640 1 -- 172.21.15.77:0/3403469127 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd16c003e60 con 0x7fd1841014b0 2024-05-27T08:38:13.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.921+0000 7fd182d9d640 1 -- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd1700b28f0 msgr2=0x7fd1700b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:13.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.921+0000 7fd182d9d640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd1700b28f0 0x7fd1700b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:38:13.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:13.955+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fd18402dba0 con 0x7fd1841014b0 2024-05-27T08:38:13.986 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:13.694+0000 7ff4526bc200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T08:38:14.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.008+0000 7fd180d99640 1 -- 172.21.15.77:0/3403469127 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7fd16c0ca420 con 0x7fd1841014b0 2024-05-27T08:38:14.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd1700b28f0 msgr2=0x7fd1700b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:38:14.011 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd1700b28f0 0x7fd1700b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1841014b0 msgr2=0x7fd184199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1841014b0 0x7fd184199160 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd16c005940 tx=0x7fd16c0042f0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 shutdown_connections 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7fd1700b28f0 0x7fd1700b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd184104080 0x7fd1841a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1841014b0 0x7fd184199160 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 --2- 172.21.15.77:0/3403469127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd184100af0 0x7fd184198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 >> 172.21.15.77:0/3403469127 conn(0x7fd1840fc6b0 msgr2=0x7fd184107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 shutdown_connections 2024-05-27T08:38:14.012 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.010+0000 7fd1890c8640 1 -- 172.21.15.77:0/3403469127 wait complete. 2024-05-27T08:38:14.015 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:13.737+0000 7f853e769200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T08:38:14.023 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-27T08:38:14.034 INFO:teuthology.run_tasks:Running task exec... 2024-05-27T08:38:14.042 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T08:38:14.043 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:38:14.043 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 -- 172.21.15.77:0/937689760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 msgr2=0x7f3aac109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/937689760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 0x7f3aac109df0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009ca0 tx=0x7f3aa402a8c0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 -- 172.21.15.77:0/937689760 shutdown_connections 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/937689760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac102ba0 0x7f3aac103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/937689760 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 0x7f3aac109df0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/937689760 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aac1019b0 0x7f3aac101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 -- 172.21.15.77:0/937689760 >> 172.21.15.77:0/937689760 conn(0x7f3aac0fcf20 msgr2=0x7f3aac0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 -- 172.21.15.77:0/937689760 shutdown_connections 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.144+0000 7f3ab0c09640 1 -- 172.21.15.77:0/937689760 wait complete. 2024-05-27T08:38:14.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3ab0c09640 1 Processor -- start 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3ab0c09640 1 -- start start 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3ab0c09640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aac1019b0 0x7f3aac06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3ab0c09640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac102ba0 0x7f3aac19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3ab0c09640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 0x7f3aac19ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3ab0c09640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac10d4c0 con 0x7f3aac103560 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3ab0c09640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac10d340 con 0x7f3aac1019b0 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3ab0c09640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3aac10d640 con 0x7f3aac102ba0 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3aaad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 0x7f3aac19ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3aaad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 0x7f3aac19ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48924/0 (socket says 172.21.15.77:48924) 2024-05-27T08:38:14.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3aaad9d640 1 -- 172.21.15.77:0/1530238311 learned_addr learned my addr 172.21.15.77:0/1530238311 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3aa1d9b640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac102ba0 0x7f3aac19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.145+0000 7f3aaa59c640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aac1019b0 0x7f3aac06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aaad9d640 1 -- 172.21.15.77:0/1530238311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac102ba0 msgr2=0x7f3aac19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aaad9d640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac102ba0 0x7f3aac19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aaad9d640 1 -- 172.21.15.77:0/1530238311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aac1019b0 msgr2=0x7f3aac06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aaad9d640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aac1019b0 0x7f3aac06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aaad9d640 1 -- 172.21.15.77:0/1530238311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aac1a0810 con 0x7f3aac103560 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aaa59c640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aac1019b0 0x7f3aac06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aaad9d640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 0x7f3aac19ff50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40097b0 tx=0x7f3aa4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aa3fff640 1 -- 172.21.15.77:0/1530238311 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4003bf0 con 0x7f3aac103560 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aac1a0aa0 con 0x7f3aac103560 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aa3fff640 1 -- 172.21.15.77:0/1530238311 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa4003d90 con 0x7f3aac103560 2024-05-27T08:38:14.150 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3aa3fff640 1 -- 172.21.15.77:0/1530238311 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa402fc70 con 0x7f3aac103560 2024-05-27T08:38:14.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.146+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aac1a1040 con 0x7f3aac103560 2024-05-27T08:38:14.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.147+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aac108cd0 con 0x7f3aac103560 2024-05-27T08:38:14.151 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.149+0000 7f3aa3fff640 1 -- 172.21.15.77:0/1530238311 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f3aa4004b40 con 0x7f3aac103560 2024-05-27T08:38:14.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.149+0000 7f3aa3fff640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a980b28f0 0x7f3a980b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.149+0000 7f3aaa59c640 1 -- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a980b28f0 msgr2=0x7f3a980b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:14.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.149+0000 7f3aa3fff640 1 -- 172.21.15.77:0/1530238311 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f3aa41039a0 con 0x7f3aac103560 2024-05-27T08:38:14.152 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.149+0000 7f3aaa59c640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a980b28f0 0x7f3a980b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:38:14.154 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.153+0000 7f3aa3fff640 1 -- 172.21.15.77:0/1530238311 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3aa40cb6c0 con 0x7f3aac103560 2024-05-27T08:38:14.302 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:14 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3403469127' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-27T08:38:14.303 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:13.923+0000 7f1daa106200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-27T08:38:14.303 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:14.301+0000 7f1daa106200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T08:38:14.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.340+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f3aac063650 con 0x7f3aac103560 2024-05-27T08:38:14.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.347+0000 7f3aa3fff640 1 -- 172.21.15.77:0/1530238311 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f3aa40cafc0 con 0x7f3aac103560 2024-05-27T08:38:14.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a980b28f0 msgr2=0x7f3a980b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:38:14.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3aaa59c640 1 -- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a980b28f0 msgr2=0x7f3a980b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:14.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3aaa59c640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a980b28f0 0x7f3a980b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:38:14.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a980b28f0 0x7f3a980b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 msgr2=0x7f3aac19ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 0x7f3aac19ff50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40097b0 tx=0x7f3aa4031840 comp rx=0 tx=0).stop 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 shutdown_connections 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f3a980b28f0 0x7f3a980b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3aac102ba0 0x7f3aac19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3aac103560 0x7f3aac19ff50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 --2- 172.21.15.77:0/1530238311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3aac1019b0 0x7f3aac06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 >> 172.21.15.77:0/1530238311 conn(0x7f3aac0fcf20 msgr2=0x7f3aac105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 shutdown_connections 2024-05-27T08:38:14.352 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.350+0000 7f3ab0c09640 1 -- 172.21.15.77:0/1530238311 wait complete. 2024-05-27T08:38:14.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:14 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3403469127' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-27T08:38:14.356 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:14.094+0000 7f853e769200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T08:38:14.356 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:14.189+0000 7f853e769200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T08:38:14.362 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-27T08:38:14.396 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:14 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3403469127' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-27T08:38:14.397 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:14.055+0000 7ff4526bc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-27T08:38:14.397 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:14.149+0000 7ff4526bc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.465+0000 7f34f2aa7640 1 -- 172.21.15.77:0/4002518579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec104080 msgr2=0x7f34ec1044a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.465+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/4002518579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec104080 0x7f34ec1044a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009900 tx=0x7f34e002a410 comp rx=0 tx=0).stop 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 -- 172.21.15.77:0/4002518579 shutdown_connections 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/4002518579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34ec100af0 0x7f34ec100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/4002518579 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec1014b0 0x7f34ec10b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/4002518579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec104080 0x7f34ec1044a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 -- 172.21.15.77:0/4002518579 >> 172.21.15.77:0/4002518579 conn(0x7f34ec0fc6b0 msgr2=0x7f34ec0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 -- 172.21.15.77:0/4002518579 shutdown_connections 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 -- 172.21.15.77:0/4002518579 wait complete. 2024-05-27T08:38:14.469 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 Processor -- start 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 -- start start 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec100af0 0x7f34ec198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec1014b0 0x7f34ec1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34ec104080 0x7f34ec1a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec10e5b0 con 0x7f34ec100af0 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec10e430 con 0x7f34ec1014b0 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f2aa7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec10e730 con 0x7f34ec104080 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f0843640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec100af0 0x7f34ec198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f0843640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec100af0 0x7f34ec198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48932/0 (socket says 172.21.15.77:48932) 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34f0843640 1 -- 172.21.15.77:0/1778347879 learned_addr learned my addr 172.21.15.77:0/1778347879 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.466+0000 7f34ebfff640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec1014b0 0x7f34ec1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.470 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f1044640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34ec104080 0x7f34ec1a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f0843640 1 -- 172.21.15.77:0/1778347879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34ec104080 msgr2=0x7f34ec1a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f0843640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34ec104080 0x7f34ec1a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f0843640 1 -- 172.21.15.77:0/1778347879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec1014b0 msgr2=0x7f34ec1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f0843640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec1014b0 0x7f34ec1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f0843640 1 -- 172.21.15.77:0/1778347879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec1a08c0 con 0x7f34ec100af0 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f0843640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec100af0 0x7f34ec198c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34e0006010 tx=0x7f34e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34e9ffb640 1 -- 172.21.15.77:0/1778347879 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e003f030 con 0x7f34ec100af0 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec1a0b50 con 0x7f34ec100af0 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34e9ffb640 1 -- 172.21.15.77:0/1778347879 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34e0003cb0 con 0x7f34ec100af0 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34e9ffb640 1 -- 172.21.15.77:0/1778347879 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0035070 con 0x7f34ec100af0 2024-05-27T08:38:14.471 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.467+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec1a1040 con 0x7f34ec100af0 2024-05-27T08:38:14.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.468+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ec109f90 con 0x7f34ec100af0 2024-05-27T08:38:14.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.470+0000 7f34e9ffb640 1 -- 172.21.15.77:0/1778347879 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f34e0042030 con 0x7f34ec100af0 2024-05-27T08:38:14.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.471+0000 7f34e9ffb640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f34dc0b28f0 0x7f34dc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.471+0000 7f34ebfff640 1 -- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f34dc0b28f0 msgr2=0x7f34dc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:14.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.471+0000 7f34e9ffb640 1 -- 172.21.15.77:0/1778347879 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f34e0102c00 con 0x7f34ec100af0 2024-05-27T08:38:14.472 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.471+0000 7f34ebfff640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f34dc0b28f0 0x7f34dc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:38:14.477 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.476+0000 7f34e9ffb640 1 -- 172.21.15.77:0/1778347879 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f34e00ca2d0 con 0x7f34ec100af0 2024-05-27T08:38:14.629 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:14.405+0000 7f1daa106200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-27T08:38:14.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.668+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f34ec02dba0 con 0x7f34ec100af0 2024-05-27T08:38:14.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.672+0000 7f34ebfff640 1 -- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f34dc0b28f0 msgr2=0x7f34dc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:14.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.672+0000 7f34ebfff640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f34dc0b28f0 0x7f34dc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:38:14.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.672+0000 7f34e9ffb640 1 -- 172.21.15.77:0/1778347879 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f34e00ce5d0 con 0x7f34ec100af0 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.674+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f34dc0b28f0 msgr2=0x7f34dc0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.674+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f34dc0b28f0 0x7f34dc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.674+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec100af0 msgr2=0x7f34ec198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.674+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec100af0 0x7f34ec198c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34e0006010 tx=0x7f34e00042f0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.675+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 shutdown_connections 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.675+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f34dc0b28f0 0x7f34dc0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.675+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34ec104080 0x7f34ec1a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.675+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f34ec1014b0 0x7f34ec1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.675+0000 7f34f2aa7640 1 --2- 172.21.15.77:0/1778347879 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34ec100af0 0x7f34ec198c80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.675+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 >> 172.21.15.77:0/1778347879 conn(0x7f34ec0fc6b0 msgr2=0x7f34ec107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:14.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.675+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 shutdown_connections 2024-05-27T08:38:14.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.675+0000 7f34f2aa7640 1 -- 172.21.15.77:0/1778347879 wait complete. 2024-05-27T08:38:14.687 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-27T08:38:14.736 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:14.395+0000 7ff4526bc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T08:38:14.736 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:14.530+0000 7ff4526bc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T08:38:14.736 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:14.626+0000 7ff4526bc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T08:38:14.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 -- 172.21.15.77:0/4163750677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c103d30 msgr2=0x7ffb7c1041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.792 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 --2- 172.21.15.77:0/4163750677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c103d30 0x7ffb7c1041b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009930 tx=0x7ffb6402a420 comp rx=0 tx=0).stop 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 -- 172.21.15.77:0/4163750677 shutdown_connections 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 --2- 172.21.15.77:0/4163750677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb7c1046f0 0x7ffb7c10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 --2- 172.21.15.77:0/4163750677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c103d30 0x7ffb7c1041b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 --2- 172.21.15.77:0/4163750677 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c100d60 0x7ffb7c101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 -- 172.21.15.77:0/4163750677 >> 172.21.15.77:0/4163750677 conn(0x7ffb7c0fc6b0 msgr2=0x7ffb7c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 -- 172.21.15.77:0/4163750677 shutdown_connections 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.789+0000 7ffb827c1640 1 -- 172.21.15.77:0/4163750677 wait complete. 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb827c1640 1 Processor -- start 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb827c1640 1 -- start start 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb827c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c100d60 0x7ffb7c198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb827c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb7c103d30 0x7ffb7c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb827c1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c1046f0 0x7ffb7c1a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.793 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb827c1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10e640 con 0x7ffb7c100d60 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb827c1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10e4c0 con 0x7ffb7c1046f0 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb827c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10e7c0 con 0x7ffb7c103d30 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb7bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c100d60 0x7ffb7c198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb7bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c100d60 0x7ffb7c198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48944/0 (socket says 172.21.15.77:48944) 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb7bfff640 1 -- 172.21.15.77:0/1599204387 learned_addr learned my addr 172.21.15.77:0/1599204387 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb80d5e640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c1046f0 0x7ffb7c1a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.790+0000 7ffb73fff640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb7c103d30 0x7ffb7c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb7bfff640 1 -- 172.21.15.77:0/1599204387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb7c103d30 msgr2=0x7ffb7c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb7bfff640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb7c103d30 0x7ffb7c199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb7bfff640 1 -- 172.21.15.77:0/1599204387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c1046f0 msgr2=0x7ffb7c1a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb7bfff640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c1046f0 0x7ffb7c1a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.794 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb7bfff640 1 -- 172.21.15.77:0/1599204387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c1a0910 con 0x7ffb7c100d60 2024-05-27T08:38:14.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb80d5e640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c1046f0 0x7ffb7c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:14.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb7bfff640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c100d60 0x7ffb7c198d00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00d770 tx=0x7ffb6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:14.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb79ffb640 1 -- 172.21.15.77:0/1599204387 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c017070 con 0x7ffb7c100d60 2024-05-27T08:38:14.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c1a0c00 con 0x7ffb7c100d60 2024-05-27T08:38:14.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb79ffb640 1 -- 172.21.15.77:0/1599204387 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb6c009c30 con 0x7ffb7c100d60 2024-05-27T08:38:14.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb79ffb640 1 -- 172.21.15.77:0/1599204387 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c003cc0 con 0x7ffb7c100d60 2024-05-27T08:38:14.795 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.791+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c1a11a0 con 0x7ffb7c100d60 2024-05-27T08:38:14.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.792+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c109f80 con 0x7ffb7c100d60 2024-05-27T08:38:14.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.793+0000 7ffb79ffb640 1 -- 172.21.15.77:0/1599204387 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7ffb6c003e60 con 0x7ffb7c100d60 2024-05-27T08:38:14.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.794+0000 7ffb79ffb640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffb680b28f0 0x7ffb680b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:14.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.794+0000 7ffb73fff640 1 -- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffb680b28f0 msgr2=0x7ffb680b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:14.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.794+0000 7ffb79ffb640 1 -- 172.21.15.77:0/1599204387 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7ffb6c0e1d60 con 0x7ffb7c100d60 2024-05-27T08:38:14.796 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.794+0000 7ffb73fff640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffb680b28f0 0x7ffb680b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:38:14.799 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.798+0000 7ffb79ffb640 1 -- 172.21.15.77:0/1599204387 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffb6c0a9a80 con 0x7ffb7c100d60 2024-05-27T08:38:14.856 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:14.436+0000 7f853e769200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T08:38:14.857 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:14.571+0000 7f853e769200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T08:38:14.857 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:14.667+0000 7f853e769200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T08:38:14.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.988+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7ffb7c063760 con 0x7ffb7c100d60 2024-05-27T08:38:14.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.992+0000 7ffb79ffb640 1 -- 172.21.15.77:0/1599204387 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7ffb6c009de0 con 0x7ffb7c100d60 2024-05-27T08:38:14.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb73fff640 1 -- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffb680b28f0 msgr2=0x7ffb680b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:14.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb73fff640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffb680b28f0 0x7ffb680b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:38:14.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffb680b28f0 msgr2=0x7ffb680b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:38:14.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffb680b28f0 0x7ffb680b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c100d60 msgr2=0x7ffb7c198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c100d60 0x7ffb7c198d00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00d770 tx=0x7ffb6c00dc30 comp rx=0 tx=0).stop 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 shutdown_connections 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7ffb680b28f0 0x7ffb680b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb7c103d30 0x7ffb7c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb7c1046f0 0x7ffb7c1a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 --2- 172.21.15.77:0/1599204387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb7c100d60 0x7ffb7c198d00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 >> 172.21.15.77:0/1599204387 conn(0x7ffb7c0fc6b0 msgr2=0x7ffb7c107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 shutdown_connections 2024-05-27T08:38:14.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:14.995+0000 7ffb827c1640 1 -- 172.21.15.77:0/1599204387 wait complete. 2024-05-27T08:38:15.007 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:14.669+0000 7f1daa106200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-27T08:38:15.007 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:14.816+0000 7f1daa106200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-27T08:38:15.007 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:14.918+0000 7f1daa106200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-27T08:38:15.008 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 -- 172.21.15.77:0/2612349046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0104670 msgr2=0x7f02f010af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 --2- 172.21.15.77:0/2612349046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0104670 0x7f02f010af00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009ca0 tx=0x7f02e802a8c0 comp rx=0 tx=0).stop 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 -- 172.21.15.77:0/2612349046 shutdown_connections 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 --2- 172.21.15.77:0/2612349046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02f0103cb0 0x7f02f0104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 --2- 172.21.15.77:0/2612349046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0104670 0x7f02f010af00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 --2- 172.21.15.77:0/2612349046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f0102ac0 0x7f02f0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 -- 172.21.15.77:0/2612349046 >> 172.21.15.77:0/2612349046 conn(0x7f02f00fe010 msgr2=0x7f02f0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 -- 172.21.15.77:0/2612349046 shutdown_connections 2024-05-27T08:38:15.116 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.114+0000 7f02f5254640 1 -- 172.21.15.77:0/2612349046 wait complete. 2024-05-27T08:38:15.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.115+0000 7f02f5254640 1 Processor -- start 2024-05-27T08:38:15.118 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.115+0000 7f02f5254640 1 -- start start 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02f5254640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0102ac0 0x7f02f0076800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02f5254640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f0103cb0 0x7f02f0074e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02f5254640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02f0104670 0x7f02f0075380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02f5254640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02f010e640 con 0x7f02f0102ac0 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02f5254640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f02f010e4c0 con 0x7f02f0103cb0 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02f5254640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f02f010e7c0 con 0x7f02f0104670 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0102ac0 0x7f02f0076800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0102ac0 0x7f02f0076800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48956/0 (socket says 172.21.15.77:48956) 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 -- 172.21.15.77:0/2838762318 learned_addr learned my addr 172.21.15.77:0/2838762318 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02ee59c640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f0103cb0 0x7f02f0074e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02ef59e640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02f0104670 0x7f02f0075380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 -- 172.21.15.77:0/2838762318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02f0104670 msgr2=0x7f02f0075380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:15.119 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02f0104670 0x7f02f0075380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 -- 172.21.15.77:0/2838762318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f0103cb0 msgr2=0x7f02f0074e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02f0103cb0 0x7f02f0074e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 -- 172.21.15.77:0/2838762318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f0075c10 con 0x7f02f0102ac0 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02ef59e640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02f0104670 0x7f02f0075380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02eed9d640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0102ac0 0x7f02f0076800 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f02e000d950 tx=0x7f02e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0017020 con 0x7f02f0102ac0 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f01a2d50 con 0x7f02f0102ac0 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e0009c30 con 0x7f02f0102ac0 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e000f6b0 con 0x7f02f0102ac0 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.116+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f01a3200 con 0x7f02f0102ac0 2024-05-27T08:38:15.120 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.117+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f0109de0 con 0x7f02f0102ac0 2024-05-27T08:38:15.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.119+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f02e001a020 con 0x7f02f0102ac0 2024-05-27T08:38:15.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.120+0000 7f02e7fff640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 0x7f02dc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:15.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.120+0000 7f02ee59c640 1 -- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 msgr2=0x7f02dc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:15.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.120+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8240+0+0 (secure 0 0 0) 0x7f02e00e22e0 con 0x7f02f0102ac0 2024-05-27T08:38:15.121 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.120+0000 7f02ee59c640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 0x7f02dc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-27T08:38:15.124 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.123+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f02e0009de0 con 0x7f02f0102ac0 2024-05-27T08:38:15.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:14.949+0000 7ff4526bc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T08:38:15.236 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:15.042+0000 7ff4526bc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T08:38:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:15 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3403469127' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-27T08:38:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:15 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:38:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:15 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1530238311' entity='client.admin' 2024-05-27T08:38:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:15 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1778347879' entity='client.admin' 2024-05-27T08:38:15.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:15 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1599204387' entity='client.admin' 2024-05-27T08:38:15.263 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:15 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3403469127' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-27T08:38:15.263 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:15 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:38:15.263 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:15 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1530238311' entity='client.admin' 2024-05-27T08:38:15.263 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:15 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1778347879' entity='client.admin' 2024-05-27T08:38:15.263 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:15 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1599204387' entity='client.admin' 2024-05-27T08:38:15.263 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:15.261+0000 7f1daa106200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T08:38:15.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.321+0000 7f02ee59c640 1 -- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 msgr2=0x7f02dc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:15.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.321+0000 7f02ee59c640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 0x7f02dc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-27T08:38:15.332 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.331+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 --> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f02f0062e90 con 0x7f02dc0b28f0 2024-05-27T08:38:15.356 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:15.000+0000 7f853e769200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-27T08:38:15.356 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:15.095+0000 7f853e769200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T08:38:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:15 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3403469127' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-27T08:38:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:15 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:38:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:15 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1530238311' entity='client.admin' 2024-05-27T08:38:15.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:15 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1778347879' entity='client.admin' 2024-05-27T08:38:15.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:15 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1599204387' entity='client.admin' 2024-05-27T08:38:15.629 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:15.360+0000 7f1daa106200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-27T08:38:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.722+0000 7f02ee59c640 1 -- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 msgr2=0x7f02dc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.77:6800/2118880698 2024-05-27T08:38:15.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:15.722+0000 7f02ee59c640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 0x7f02dc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-27T08:38:15.986 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:15.643+0000 7ff4526bc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T08:38:15.986 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:38:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y[37422]: 2024-05-27T08:38:15.858+0000 7ff4526bc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T08:38:16.018 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:15.698+0000 7f853e769200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T08:38:16.019 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:38:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z[38322]: 2024-05-27T08:38:15.915+0000 7f853e769200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T08:38:16.035 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:16.033+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f02e00a5b90 con 0x7f02f0102ac0 2024-05-27T08:38:16.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:16.224+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7f02e00dfe60 con 0x7f02f0102ac0 2024-05-27T08:38:16.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:16.225+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 msgr2=0x7f02dc0b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-27T08:38:16.225 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:16.225+0000 7f02e7fff640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/2118880698,v1:172.21.15.77:6801/2118880698] conn(0x7f02dc0b28f0 0x7f02dc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: Standby manager daemon y restarted 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: Standby manager daemon y started 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: Standby manager daemon z restarted 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: Standby manager daemon z started 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T08:38:16.352 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:38:16.353 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T08:38:16.353 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:16 smithi179 ceph-mon[36481]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:38:16.353 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:15.974+0000 7f1daa106200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-27T08:38:16.353 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.208+0000 7f1daa106200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: Standby manager daemon y restarted 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: Standby manager daemon y started 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: Standby manager daemon z restarted 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: Standby manager daemon z started 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T08:38:16.354 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:16 smithi092 ceph-mon[36049]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: Standby manager daemon y restarted 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: Standby manager daemon y started 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.92:0/276531358' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: Standby manager daemon z restarted 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: Standby manager daemon z started 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-27T08:38:16.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:16 smithi077 ceph-mon[32007]: from='mgr.? 172.21.15.179:0/3176327862' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d089bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d089bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d089bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d089bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d089bc700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d0b9c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d0b9c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d0b9c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d0b9c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:16.632 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:38:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:38:16.455+0000 7f1d0b9c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:38:17.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.228+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mon.0 v2:172.21.15.77:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f02e00109a0 con 0x7f02f0102ac0 2024-05-27T08:38:17.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.229+0000 7f02e7fff640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f02dc039d90 0x7f02dc03a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.229+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f02e000d130 con 0x7f02dc039d90 2024-05-27T08:38:17.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.232+0000 7f02ee59c640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f02dc039d90 0x7f02dc03a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.234+0000 7f02ee59c640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f02dc039d90 0x7f02dc03a170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f02f0076570 tx=0x7f02d8005f90 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:17.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.297+0000 7f02e7fff640 1 -- 172.21.15.77:0/2838762318 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f02e000d130 con 0x7f02dc039d90 2024-05-27T08:38:17.299 INFO:teuthology.orchestra.run.smithi077.stdout:Schedule set for path / 2024-05-27T08:38:17.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f02dc039d90 msgr2=0x7f02dc03a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02f5254640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f02dc039d90 0x7f02dc03a170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f02f0076570 tx=0x7f02d8005f90 comp rx=0 tx=0).stop 2024-05-27T08:38:17.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0102ac0 msgr2=0x7f02f0076800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02f5254640 1 --2- 172.21.15.77:0/2838762318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f0102ac0 0x7f02f0076800 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f02e000d950 tx=0x7f02e000de10 comp rx=0 tx=0).stop 2024-05-27T08:38:17.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02ef59e640 1 -- 172.21.15.77:0/2838762318 reap_dead start 2024-05-27T08:38:17.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 shutdown_connections 2024-05-27T08:38:17.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 >> 172.21.15.77:0/2838762318 conn(0x7f02f00fe010 msgr2=0x7f02f0106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:17.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 shutdown_connections 2024-05-27T08:38:17.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.299+0000 7f02f5254640 1 -- 172.21.15.77:0/2838762318 wait complete. 2024-05-27T08:38:17.311 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-27T08:38:17.311 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: Active manager daemon x restarted 2024-05-27T08:38:17.312 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: Activating manager daemon x 2024-05-27T08:38:17.312 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T08:38:17.312 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: mgrmap e21: x(active, starting, since 0.0164915s), standbys: z, y 2024-05-27T08:38:17.312 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:38:17.312 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:38:17.312 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:38:17.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T08:38:17.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T08:38:17.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:38:17.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:38:17.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:38:17.313 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:38:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:38:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:38:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:38:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:38:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:38:17.314 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:38:17.315 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T08:38:17.315 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T08:38:17.315 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T08:38:17.315 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:38:17.315 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:38:17.315 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:38:17.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:38:17.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:38:17.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:38:17.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:38:17.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:38:17.316 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:38:17.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:38:17.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:38:17.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:38:17.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T08:38:17.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T08:38:17.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T08:38:17.317 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: Manager daemon x is now available 2024-05-27T08:38:17.318 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:38:17.318 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T08:38:17.318 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T08:38:17.318 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: [27/May/2024:08:38:16] ENGINE Bus STARTING 2024-05-27T08:38:17.318 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: [27/May/2024:08:38:16] ENGINE Serving on https://172.21.15.77:7150 2024-05-27T08:38:17.318 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: [27/May/2024:08:38:16] ENGINE Serving on http://172.21.15.77:8765 2024-05-27T08:38:17.319 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:17 smithi077 ceph-mon[32007]: [27/May/2024:08:38:16] ENGINE Bus STARTED 2024-05-27T08:38:17.320 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-27T08:38:17.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-27T08:38:17.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: Active manager daemon x restarted 2024-05-27T08:38:17.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: Activating manager daemon x 2024-05-27T08:38:17.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T08:38:17.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: mgrmap e21: x(active, starting, since 0.0164915s), standbys: z, y 2024-05-27T08:38:17.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:38:17.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:38:17.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:38:17.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T08:38:17.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T08:38:17.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:38:17.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:38:17.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:38:17.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:38:17.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:38:17.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:38:17.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:38:17.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:38:17.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:38:17.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:38:17.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T08:38:17.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T08:38:17.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T08:38:17.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:38:17.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:38:17.361 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:38:17.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:38:17.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:38:17.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:38:17.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:38:17.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:38:17.362 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:38:17.363 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:38:17.363 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:38:17.363 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:38:17.363 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T08:38:17.363 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T08:38:17.363 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T08:38:17.364 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: Manager daemon x is now available 2024-05-27T08:38:17.364 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:38:17.364 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T08:38:17.364 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T08:38:17.364 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: [27/May/2024:08:38:16] ENGINE Bus STARTING 2024-05-27T08:38:17.364 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: [27/May/2024:08:38:16] ENGINE Serving on https://172.21.15.77:7150 2024-05-27T08:38:17.365 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: [27/May/2024:08:38:16] ENGINE Serving on http://172.21.15.77:8765 2024-05-27T08:38:17.365 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:17 smithi179 ceph-mon[36481]: [27/May/2024:08:38:16] ENGINE Bus STARTED 2024-05-27T08:38:17.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 -- 172.21.15.77:0/1064068341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b4102ba0 msgr2=0x7f09b4103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1064068341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b4102ba0 0x7f09b4103020 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009ca0 tx=0x7f09a802a8c0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 -- 172.21.15.77:0/1064068341 shutdown_connections 2024-05-27T08:38:17.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1064068341 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4103560 0x7f09b4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1064068341 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b4102ba0 0x7f09b4103020 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1064068341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b41019b0 0x7f09b4101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 -- 172.21.15.77:0/1064068341 >> 172.21.15.77:0/1064068341 conn(0x7f09b40fcf20 msgr2=0x7f09b40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 -- 172.21.15.77:0/1064068341 shutdown_connections 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 -- 172.21.15.77:0/1064068341 wait complete. 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 Processor -- start 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.424+0000 7f09bab0a640 1 -- start start 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09bab0a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b41019b0 0x7f09b406ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09bab0a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b4102ba0 0x7f09b419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09bab0a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4103560 0x7f09b41a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09bab0a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09b410d4c0 con 0x7f09b41019b0 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09bab0a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f09b410d340 con 0x7f09b4103560 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09bab0a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f09b410d640 con 0x7f09b4102ba0 2024-05-27T08:38:17.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b41019b0 0x7f09b406ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b41019b0 0x7f09b406ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49100/0 (socket says 172.21.15.77:49100) 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 -- 172.21.15.77:0/1989708516 learned_addr learned my addr 172.21.15.77:0/1989708516 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b3fff640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b4102ba0 0x7f09b419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b90a7640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4103560 0x7f09b41a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 -- 172.21.15.77:0/1989708516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b4102ba0 msgr2=0x7f09b419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b4102ba0 0x7f09b419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 -- 172.21.15.77:0/1989708516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4103560 msgr2=0x7f09b41a0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4103560 0x7f09b41a0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 -- 172.21.15.77:0/1989708516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b41a08f0 con 0x7f09b41019b0 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b88a6640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b41019b0 0x7f09b406ac40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f09a400d770 tx=0x7f09a400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b1ffb640 1 -- 172.21.15.77:0/1989708516 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4017070 con 0x7f09b41019b0 2024-05-27T08:38:17.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b1ffb640 1 -- 172.21.15.77:0/1989708516 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a4009c30 con 0x7f09b41019b0 2024-05-27T08:38:17.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b41a0be0 con 0x7f09b41019b0 2024-05-27T08:38:17.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.425+0000 7f09b1ffb640 1 -- 172.21.15.77:0/1989708516 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4003cc0 con 0x7f09b41019b0 2024-05-27T08:38:17.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.426+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b41a1100 con 0x7f09b41019b0 2024-05-27T08:38:17.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.427+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b4108d90 con 0x7f09b41019b0 2024-05-27T08:38:17.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.428+0000 7f09b1ffb640 1 -- 172.21.15.77:0/1989708516 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f09a4003e60 con 0x7f09b41019b0 2024-05-27T08:38:17.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.429+0000 7f09b1ffb640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f099c0b2690 0x7f099c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.429+0000 7f09b3fff640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f099c0b2690 0x7f099c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.429+0000 7f09b1ffb640 1 -- 172.21.15.77:0/1989708516 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f09a40e18d0 con 0x7f09b41019b0 2024-05-27T08:38:17.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.429+0000 7f09b3fff640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f099c0b2690 0x7f099c0b4b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09b406a9b0 tx=0x7f09a8005dc0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:17.434 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.433+0000 7f09b1ffb640 1 -- 172.21.15.77:0/1989708516 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09a40a97d0 con 0x7f09b41019b0 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: Active manager daemon x restarted 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: Activating manager daemon x 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: osdmap e86: 12 total, 12 up, 12 in 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: mgrmap e21: x(active, starting, since 0.0164915s), standbys: z, y 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-27T08:38:17.465 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-27T08:38:17.466 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-27T08:38:17.467 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: Manager daemon x is now available 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: [27/May/2024:08:38:16] ENGINE Bus STARTING 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: [27/May/2024:08:38:16] ENGINE Serving on https://172.21.15.77:7150 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: [27/May/2024:08:38:16] ENGINE Serving on http://172.21.15.77:8765 2024-05-27T08:38:17.468 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:17 smithi092 ceph-mon[36049]: [27/May/2024:08:38:16] ENGINE Bus STARTED 2024-05-27T08:38:17.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.648+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f09b4062ff0 con 0x7f099c0b2690 2024-05-27T08:38:17.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.703+0000 7f09b1ffb640 1 -- 172.21.15.77:0/1989708516 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f09b4062ff0 con 0x7f099c0b2690 2024-05-27T08:38:17.704 INFO:teuthology.orchestra.run.smithi077.stdout:Retention added to path / 2024-05-27T08:38:17.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f099c0b2690 msgr2=0x7f099c0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f099c0b2690 0x7f099c0b4b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09b406a9b0 tx=0x7f09a8005dc0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b41019b0 msgr2=0x7f09b406ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b41019b0 0x7f09b406ac40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f09a400d770 tx=0x7f09a400dc30 comp rx=0 tx=0).stop 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 shutdown_connections 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f099c0b2690 0x7f099c0b4b50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09b4102ba0 0x7f09b419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f09b4103560 0x7f09b41a0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 --2- 172.21.15.77:0/1989708516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09b41019b0 0x7f09b406ac40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 >> 172.21.15.77:0/1989708516 conn(0x7f09b40fcf20 msgr2=0x7f09b4105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 shutdown_connections 2024-05-27T08:38:17.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.705+0000 7f09bab0a640 1 -- 172.21.15.77:0/1989708516 wait complete. 2024-05-27T08:38:17.717 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T08:38:17.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 -- 172.21.15.77:0/2732575768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 msgr2=0x7fbd40103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/2732575768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 0x7fbd40103060 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009900 tx=0x7fbd3002a410 comp rx=0 tx=0).stop 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 -- 172.21.15.77:0/2732575768 shutdown_connections 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/2732575768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd401035a0 0x7fbd40109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/2732575768 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 0x7fbd40103060 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/2732575768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd401019f0 0x7fbd40101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 -- 172.21.15.77:0/2732575768 >> 172.21.15.77:0/2732575768 conn(0x7fbd400fcf60 msgr2=0x7fbd400ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 -- 172.21.15.77:0/2732575768 shutdown_connections 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.818+0000 7fbd46e6b640 1 -- 172.21.15.77:0/2732575768 wait complete. 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd46e6b640 1 Processor -- start 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd46e6b640 1 -- start start 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd46e6b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd401019f0 0x7fbd4006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd46e6b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 0x7fbd4019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd46e6b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd401035a0 0x7fbd4019fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd46e6b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4010d570 con 0x7fbd40102be0 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd46e6b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4010d3f0 con 0x7fbd401035a0 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd46e6b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4010d6f0 con 0x7fbd401019f0 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 0x7fbd4019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 0x7fbd4019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49116/0 (socket says 172.21.15.77:49116) 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 -- 172.21.15.77:0/98627806 learned_addr learned my addr 172.21.15.77:0/98627806 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd44c07640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd401019f0 0x7fbd4006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd45408640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd401035a0 0x7fbd4019fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 -- 172.21.15.77:0/98627806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd401019f0 msgr2=0x7fbd4006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd401019f0 0x7fbd4006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 -- 172.21.15.77:0/98627806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd401035a0 msgr2=0x7fbd4019fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd401035a0 0x7fbd4019fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:17.823 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 -- 172.21.15.77:0/98627806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd401a08b0 con 0x7fbd40102be0 2024-05-27T08:38:17.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.819+0000 7fbd37fff640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 0x7fbd4019d9b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd30005ab0 tx=0x7fbd30003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:17.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.820+0000 7fbd35ffb640 1 -- 172.21.15.77:0/98627806 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3003f070 con 0x7fbd40102be0 2024-05-27T08:38:17.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.820+0000 7fbd35ffb640 1 -- 172.21.15.77:0/98627806 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd30031c50 con 0x7fbd40102be0 2024-05-27T08:38:17.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.820+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd401a0b40 con 0x7fbd40102be0 2024-05-27T08:38:17.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.820+0000 7fbd35ffb640 1 -- 172.21.15.77:0/98627806 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30035030 con 0x7fbd40102be0 2024-05-27T08:38:17.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.820+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd401a1080 con 0x7fbd40102be0 2024-05-27T08:38:17.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.821+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd40108d10 con 0x7fbd40102be0 2024-05-27T08:38:17.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.822+0000 7fbd35ffb640 1 -- 172.21.15.77:0/98627806 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fbd30042070 con 0x7fbd40102be0 2024-05-27T08:38:17.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.823+0000 7fbd35ffb640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbd280b2710 0x7fbd280b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:17.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.823+0000 7fbd44c07640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbd280b2710 0x7fbd280b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:17.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.823+0000 7fbd35ffb640 1 -- 172.21.15.77:0/98627806 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbd30102710 con 0x7fbd40102be0 2024-05-27T08:38:17.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.823+0000 7fbd44c07640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbd280b2710 0x7fbd280b4bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbd40102a40 tx=0x7fbd3c005f90 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:17.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:17.827+0000 7fbd35ffb640 1 -- 172.21.15.77:0/98627806 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd300ca610 con 0x7fbd40102be0 2024-05-27T08:38:18.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.087+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fbd40062e90 con 0x7fbd280b2710 2024-05-27T08:38:18.096 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.095+0000 7fbd35ffb640 1 -- 172.21.15.77:0/98627806 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fbd40062e90 con 0x7fbd280b2710 2024-05-27T08:38:18.096 INFO:teuthology.orchestra.run.smithi077.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T08:38:17", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbd280b2710 msgr2=0x7fbd280b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbd280b2710 0x7fbd280b4bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbd40102a40 tx=0x7fbd3c005f90 comp rx=0 tx=0).stop 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 msgr2=0x7fbd4019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 0x7fbd4019d9b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbd30005ab0 tx=0x7fbd30003b90 comp rx=0 tx=0).stop 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 shutdown_connections 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbd280b2710 0x7fbd280b4bd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd401035a0 0x7fbd4019fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd40102be0 0x7fbd4019d9b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.099 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 --2- 172.21.15.77:0/98627806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd401019f0 0x7fbd4006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.097+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 >> 172.21.15.77:0/98627806 conn(0x7fbd400fcf60 msgr2=0x7fbd40105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:18.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.098+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 shutdown_connections 2024-05-27T08:38:18.100 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.098+0000 7fbd46e6b640 1 -- 172.21.15.77:0/98627806 wait complete. 2024-05-27T08:38:18.112 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T08:38:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 -- 172.21.15.77:0/4132892161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 msgr2=0x7f19a0101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:18.226 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 --2- 172.21.15.77:0/4132892161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 0x7f19a0101e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1990009900 tx=0x7f199002a410 comp rx=0 tx=0).stop 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 -- 172.21.15.77:0/4132892161 shutdown_connections 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 --2- 172.21.15.77:0/4132892161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19a0102be0 0x7f19a0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 --2- 172.21.15.77:0/4132892161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19a01035a0 0x7f19a0109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 --2- 172.21.15.77:0/4132892161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 0x7f19a0101e10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 -- 172.21.15.77:0/4132892161 >> 172.21.15.77:0/4132892161 conn(0x7f19a00fcf60 msgr2=0x7f19a00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 -- 172.21.15.77:0/4132892161 shutdown_connections 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.224+0000 7f19a5a28640 1 -- 172.21.15.77:0/4132892161 wait complete. 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f19a5a28640 1 Processor -- start 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f19a5a28640 1 -- start start 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f19a5a28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 0x7f19a0198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f19a5a28640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19a0102be0 0x7f19a0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:18.227 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f19a5a28640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19a01035a0 0x7f19a01a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f19a5a28640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f19a010e6a0 con 0x7f19a01019f0 2024-05-27T08:38:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f19a5a28640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f19a010e520 con 0x7f19a01035a0 2024-05-27T08:38:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f19a5a28640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f19a010e820 con 0x7f19a0102be0 2024-05-27T08:38:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f199f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19a01035a0 0x7f19a01a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f199f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19a01035a0 0x7f19a01a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:54634/0 (socket says 172.21.15.77:54634) 2024-05-27T08:38:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f199f7fe640 1 -- 172.21.15.77:0/3025824725 learned_addr learned my addr 172.21.15.77:0/3025824725 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f1997fff640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19a0102be0 0x7f19a0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:18.228 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.225+0000 7f199effd640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 0x7f19a0198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.226+0000 7f199effd640 1 -- 172.21.15.77:0/3025824725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19a0102be0 msgr2=0x7f19a0199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.226+0000 7f199effd640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19a0102be0 0x7f19a0199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.226+0000 7f199effd640 1 -- 172.21.15.77:0/3025824725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19a01035a0 msgr2=0x7f19a01a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.226+0000 7f199effd640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19a01035a0 0x7f19a01a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.226+0000 7f199effd640 1 -- 172.21.15.77:0/3025824725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a01a08d0 con 0x7f19a01019f0 2024-05-27T08:38:18.229 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.226+0000 7f199effd640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 0x7f19a0198ce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f199002a930 tx=0x7f1990003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:18.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.239+0000 7f19a4a26640 1 -- 172.21.15.77:0/3025824725 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199003f070 con 0x7f19a01019f0 2024-05-27T08:38:18.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.239+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a010a7f0 con 0x7f19a01019f0 2024-05-27T08:38:18.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.239+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a010ad00 con 0x7f19a01019f0 2024-05-27T08:38:18.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.239+0000 7f19a4a26640 1 -- 172.21.15.77:0/3025824725 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1990031c50 con 0x7f19a01019f0 2024-05-27T08:38:18.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.239+0000 7f19a4a26640 1 -- 172.21.15.77:0/3025824725 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1990049060 con 0x7f19a01019f0 2024-05-27T08:38:18.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.240+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19800052c0 con 0x7f19a01019f0 2024-05-27T08:38:18.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.243+0000 7f19a4a26640 1 -- 172.21.15.77:0/3025824725 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7f1990042070 con 0x7f19a01019f0 2024-05-27T08:38:18.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.243+0000 7f19a4a26640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f198c0b2a10 0x7f198c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:18.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.244+0000 7f1997fff640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f198c0b2a10 0x7f198c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:18.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.244+0000 7f19a4a26640 1 -- 172.21.15.77:0/3025824725 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f19900c6790 con 0x7f19a01019f0 2024-05-27T08:38:18.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.244+0000 7f1997fff640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f198c0b2a10 0x7f198c0b4ed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19a019a190 tx=0x7f1988012040 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:18.251 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.250+0000 7f19a4a26640 1 -- 172.21.15.77:0/3025824725 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19900ca1a0 con 0x7f19a01019f0 2024-05-27T08:38:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:18 smithi077 ceph-mon[32007]: mgrmap e22: x(active, since 1.01995s), standbys: z, y 2024-05-27T08:38:18.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:18 smithi077 ceph-mon[32007]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:18.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:18 smithi077 ceph-mon[32007]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:18.380 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:18 smithi077 ceph-mon[32007]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:18.480 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.479+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1980000bf0 con 0x7f198c0b2a10 2024-05-27T08:38:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:18 smithi092 ceph-mon[36049]: mgrmap e22: x(active, since 1.01995s), standbys: z, y 2024-05-27T08:38:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:18 smithi092 ceph-mon[36049]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:18 smithi092 ceph-mon[36049]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:18.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:18 smithi092 ceph-mon[36049]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:18.487 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.486+0000 7f19a4a26640 1 -- 172.21.15.77:0/3025824725 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1980000bf0 con 0x7f198c0b2a10 2024-05-27T08:38:18.488 INFO:teuthology.orchestra.run.smithi077.stdout:/ 1m 6m3h 2024-05-27T08:38:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f198c0b2a10 msgr2=0x7f198c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f198c0b2a10 0x7f198c0b4ed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19a019a190 tx=0x7f1988012040 comp rx=0 tx=0).stop 2024-05-27T08:38:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 msgr2=0x7f19a0198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:18.491 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 0x7f19a0198ce0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f199002a930 tx=0x7f1990003c10 comp rx=0 tx=0).stop 2024-05-27T08:38:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 shutdown_connections 2024-05-27T08:38:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f198c0b2a10 0x7f198c0b4ed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f19a0102be0 0x7f19a0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f19a01035a0 0x7f19a01a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 --2- 172.21.15.77:0/3025824725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f19a01019f0 0x7f19a0198ce0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 >> 172.21.15.77:0/3025824725 conn(0x7f19a00fcf60 msgr2=0x7f19a0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 shutdown_connections 2024-05-27T08:38:18.492 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:18.490+0000 7f19a5a28640 1 -- 172.21.15.77:0/3025824725 wait complete. 2024-05-27T08:38:18.506 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-27T08:38:18.535 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-27T08:38:18.550 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-27T08:38:18.572 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-27T08:38:18.573 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T08:38:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:18 smithi179 ceph-mon[36481]: mgrmap e22: x(active, since 1.01995s), standbys: z, y 2024-05-27T08:38:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:18 smithi179 ceph-mon[36481]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:18 smithi179 ceph-mon[36481]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:18.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:18 smithi179 ceph-mon[36481]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:18.838 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:19.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:19.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:19.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T08:38:19.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.487 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:19 smithi092 ceph-mon[36049]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:19.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:19.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:19.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T08:38:19.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:19 smithi179 ceph-mon[36481]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:19.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:19.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:19 smithi077 ceph-mon[32007]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:38:20.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 -- 172.21.15.77:0/3002140265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f01024f0 msgr2=0x7f89f01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 --2- 172.21.15.77:0/3002140265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f01024f0 0x7f89f01103d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f89e800b0d0 tx=0x7f89e800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 -- 172.21.15.77:0/3002140265 shutdown_connections 2024-05-27T08:38:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 --2- 172.21.15.77:0/3002140265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f01024f0 0x7f89f01103d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 --2- 172.21.15.77:0/3002140265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f0109520 0x7f89f0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 --2- 172.21.15.77:0/3002140265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89f0101b30 0x7f89f0101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 -- 172.21.15.77:0/3002140265 >> 172.21.15.77:0/3002140265 conn(0x7f89f0078ab0 msgr2=0x7f89f00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 -- 172.21.15.77:0/3002140265 shutdown_connections 2024-05-27T08:38:20.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.606+0000 7f89f6d44700 1 -- 172.21.15.77:0/3002140265 wait complete. 2024-05-27T08:38:20.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f6d44700 1 Processor -- start 2024-05-27T08:38:20.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f6d44700 1 -- start start 2024-05-27T08:38:20.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f6d44700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89f0101b30 0x7f89f0199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:20.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f6d44700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f01024f0 0x7f89f019a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:20.609 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.609 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.609 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.609 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.610 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:38:20.611 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:20 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:38:20.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f6d44700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0109520 0x7f89f019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f6d44700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f89f01137e0 con 0x7f89f0109520 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f6d44700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0113660 con 0x7f89f01024f0 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f6d44700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89f0113960 con 0x7f89f0101b30 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f01024f0 0x7f89f019a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89effff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f01024f0 0x7f89f019a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:60582/0 (socket says 172.21.15.77:60582) 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89effff700 1 -- 172.21.15.77:0/1881040202 learned_addr learned my addr 172.21.15.77:0/1881040202 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.607+0000 7f89f4ae0700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89f0101b30 0x7f89f0199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89f52e1700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0109520 0x7f89f019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89effff700 1 -- 172.21.15.77:0/1881040202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89f0101b30 msgr2=0x7f89f0199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89effff700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89f0101b30 0x7f89f0199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89effff700 1 -- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0109520 msgr2=0x7f89f019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89effff700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0109520 0x7f89f019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89effff700 1 -- 172.21.15.77:0/1881040202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f019ed20 con 0x7f89f01024f0 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89f4ae0700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89f0101b30 0x7f89f0199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89effff700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f01024f0 0x7f89f019a250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00b530 tx=0x7f89dc00b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89edffb700 1 -- 172.21.15.77:0/1881040202 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc015020 con 0x7f89f01024f0 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89edffb700 1 -- 172.21.15.77:0/1881040202 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89dc011ce0 con 0x7f89f01024f0 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f019f010 con 0x7f89f01024f0 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.608+0000 7f89edffb700 1 -- 172.21.15.77:0/1881040202 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc010dd0 con 0x7f89f01024f0 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.609+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f0102c40 con 0x7f89f01024f0 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.610+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f006a230 con 0x7f89f01024f0 2024-05-27T08:38:20.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.611+0000 7f89edffb700 1 -- 172.21.15.77:0/1881040202 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7f89dc018020 con 0x7f89f01024f0 2024-05-27T08:38:20.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.612+0000 7f89edffb700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f89e00b2a70 0x7f89e00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:20.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.612+0000 7f89f4ae0700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f89e00b2a70 0x7f89e00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:20.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.612+0000 7f89edffb700 1 -- 172.21.15.77:0/1881040202 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f89dc0e1bb0 con 0x7f89f01024f0 2024-05-27T08:38:20.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.612+0000 7f89f4ae0700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f89e00b2a70 0x7f89e00b4f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f89e40123d0 tx=0x7f89e400f040 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:20.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.617+0000 7f89edffb700 1 -- 172.21.15.77:0/1881040202 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f89dc01f020 con 0x7f89f01024f0 2024-05-27T08:38:20.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-27T08:38:20.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:38:20.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:20 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:38:20.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-27T08:38:20.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:20.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:38:20.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:38:20.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:20 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:38:20.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.894+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89f0063980 con 0x7f89e00b2a70 2024-05-27T08:38:20.903 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.902+0000 7f89edffb700 1 -- 172.21.15.77:0/1881040202 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257812 (secure 0 0 0) 0x7f89f0063980 con 0x7f89e00b2a70 2024-05-27T08:38:20.907 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f89e00b2a70 msgr2=0x7f89e00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f89e00b2a70 0x7f89e00b4f30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f89e40123d0 tx=0x7f89e400f040 comp rx=0 tx=0).stop 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f01024f0 msgr2=0x7f89f019a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f01024f0 0x7f89f019a250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00b530 tx=0x7f89dc00b840 comp rx=0 tx=0).stop 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 shutdown_connections 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f89e00b2a70 0x7f89e00b4f30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f89f0109520 0x7f89f019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89f0101b30 0x7f89f0199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 --2- 172.21.15.77:0/1881040202 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f89f01024f0 0x7f89f019a250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.906+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 >> 172.21.15.77:0/1881040202 conn(0x7f89f0078ab0 msgr2=0x7f89f0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.907+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 shutdown_connections 2024-05-27T08:38:20.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:20.907+0000 7f89f6d44700 1 -- 172.21.15.77:0/1881040202 wait complete. 2024-05-27T08:38:20.910 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-05-27T08:38:21.599 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi077 to 176.1M 2024-05-27T08:38:21.599 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Unable to set osd_memory_target on smithi077 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-27T08:38:21.599 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-27T08:38:21.599 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.600 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T08:38:21.601 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.601 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.601 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.601 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.601 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.601 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.601 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.601 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:21 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:38:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi077 to 176.1M 2024-05-27T08:38:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Unable to set osd_memory_target on smithi077 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-27T08:38:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-27T08:38:21.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T08:38:21.607 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.608 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:21 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:38:21.625 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-27T08:38:20.232171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191250,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1948,"num_read_kb":1116,"num_write":504,"num_write_kb":3620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2953206,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":900,"ondisk_log_size":900,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368140,"kb_used_data":14728,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124573684,"statfs":{"total":1151940427776,"available":1151563452416,"internally_reserved":0,"allocated":15081472,"data_stored":8901636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990204"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439549+0000","last_change":"2024-05-27T08:35:07.046361+0000","last_active":"2024-05-27T08:38:16.439549+0000","last_peered":"2024-05-27T08:38:16.439549+0000","last_clean":"2024-05-27T08:38:16.439549+0000","last_became_active":"2024-05-27T08:35:07.045954+0000","last_became_peered":"2024-05-27T08:35:07.045954+0000","last_unstale":"2024-05-27T08:38:16.439549+0000","last_undegraded":"2024-05-27T08:38:16.439549+0000","last_fullsized":"2024-05-27T08:38:16.439549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:03:47.458964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439255+0000","last_change":"2024-05-27T08:35:07.043725+0000","last_active":"2024-05-27T08:38:16.439255+0000","last_peered":"2024-05-27T08:38:16.439255+0000","last_clean":"2024-05-27T08:38:16.439255+0000","last_became_active":"2024-05-27T08:35:07.043447+0000","last_became_peered":"2024-05-27T08:35:07.043447+0000","last_unstale":"2024-05-27T08:38:16.439255+0000","last_undegraded":"2024-05-27T08:38:16.439255+0000","last_fullsized":"2024-05-27T08:38:16.439255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:44.509202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439075+0000","last_change":"2024-05-27T08:35:07.048075+0000","last_active":"2024-05-27T08:38:16.439075+0000","last_peered":"2024-05-27T08:38:16.439075+0000","last_clean":"2024-05-27T08:38:16.439075+0000","last_became_active":"2024-05-27T08:35:07.047687+0000","last_became_peered":"2024-05-27T08:35:07.047687+0000","last_unstale":"2024-05-27T08:38:16.439075+0000","last_undegraded":"2024-05-27T08:38:16.439075+0000","last_fullsized":"2024-05-27T08:38:16.439075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:05:15.197231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255117+0000","last_change":"2024-05-27T08:35:07.046809+0000","last_active":"2024-05-27T08:38:16.255117+0000","last_peered":"2024-05-27T08:38:16.255117+0000","last_clean":"2024-05-27T08:38:16.255117+0000","last_became_active":"2024-05-27T08:35:07.046453+0000","last_became_peered":"2024-05-27T08:35:07.046453+0000","last_unstale":"2024-05-27T08:38:16.255117+0000","last_undegraded":"2024-05-27T08:38:16.255117+0000","last_fullsized":"2024-05-27T08:38:16.255117+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:43.209613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320175+0000","last_change":"2024-05-27T08:35:07.040242+0000","last_active":"2024-05-27T08:38:16.320175+0000","last_peered":"2024-05-27T08:38:16.320175+0000","last_clean":"2024-05-27T08:38:16.320175+0000","last_became_active":"2024-05-27T08:35:07.039860+0000","last_became_peered":"2024-05-27T08:35:07.039860+0000","last_unstale":"2024-05-27T08:38:16.320175+0000","last_undegraded":"2024-05-27T08:38:16.320175+0000","last_fullsized":"2024-05-27T08:38:16.320175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:49.212799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325893+0000","last_change":"2024-05-27T08:35:07.048176+0000","last_active":"2024-05-27T08:38:16.325893+0000","last_peered":"2024-05-27T08:38:16.325893+0000","last_clean":"2024-05-27T08:38:16.325893+0000","last_became_active":"2024-05-27T08:35:07.047860+0000","last_became_peered":"2024-05-27T08:35:07.047860+0000","last_unstale":"2024-05-27T08:38:16.325893+0000","last_undegraded":"2024-05-27T08:38:16.325893+0000","last_fullsized":"2024-05-27T08:38:16.325893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:19.459704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438445+0000","last_change":"2024-05-27T08:35:07.052285+0000","last_active":"2024-05-27T08:38:16.438445+0000","last_peered":"2024-05-27T08:38:16.438445+0000","last_clean":"2024-05-27T08:38:16.438445+0000","last_became_active":"2024-05-27T08:35:07.051970+0000","last_became_peered":"2024-05-27T08:35:07.051970+0000","last_unstale":"2024-05-27T08:38:16.438445+0000","last_undegraded":"2024-05-27T08:38:16.438445+0000","last_fullsized":"2024-05-27T08:38:16.438445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:56.543022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218781+0000","last_change":"2024-05-27T08:35:07.047527+0000","last_active":"2024-05-27T08:38:16.218781+0000","last_peered":"2024-05-27T08:38:16.218781+0000","last_clean":"2024-05-27T08:38:16.218781+0000","last_became_active":"2024-05-27T08:35:07.047204+0000","last_became_peered":"2024-05-27T08:35:07.047204+0000","last_unstale":"2024-05-27T08:38:16.218781+0000","last_undegraded":"2024-05-27T08:38:16.218781+0000","last_fullsized":"2024-05-27T08:38:16.218781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:04:43.547847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.220295+0000","last_change":"2024-05-27T08:35:07.049108+0000","last_active":"2024-05-27T08:38:16.220295+0000","last_peered":"2024-05-27T08:38:16.220295+0000","last_clean":"2024-05-27T08:38:16.220295+0000","last_became_active":"2024-05-27T08:35:07.048540+0000","last_became_peered":"2024-05-27T08:35:07.048540+0000","last_unstale":"2024-05-27T08:38:16.220295+0000","last_undegraded":"2024-05-27T08:38:16.220295+0000","last_fullsized":"2024-05-27T08:38:16.220295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:56.219362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.326127+0000","last_change":"2024-05-27T08:35:07.049775+0000","last_active":"2024-05-27T08:38:16.326127+0000","last_peered":"2024-05-27T08:38:16.326127+0000","last_clean":"2024-05-27T08:38:16.326127+0000","last_became_active":"2024-05-27T08:35:07.049466+0000","last_became_peered":"2024-05-27T08:35:07.049466+0000","last_unstale":"2024-05-27T08:38:16.326127+0000","last_undegraded":"2024-05-27T08:38:16.326127+0000","last_fullsized":"2024-05-27T08:38:16.326127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:18:30.394390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438881+0000","last_change":"2024-05-27T08:35:07.050870+0000","last_active":"2024-05-27T08:38:16.438881+0000","last_peered":"2024-05-27T08:38:16.438881+0000","last_clean":"2024-05-27T08:38:16.438881+0000","last_became_active":"2024-05-27T08:35:07.050527+0000","last_became_peered":"2024-05-27T08:35:07.050527+0000","last_unstale":"2024-05-27T08:38:16.438881+0000","last_undegraded":"2024-05-27T08:38:16.438881+0000","last_fullsized":"2024-05-27T08:38:16.438881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:56:41.477053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.254986+0000","last_change":"2024-05-27T08:35:07.052955+0000","last_active":"2024-05-27T08:38:16.254986+0000","last_peered":"2024-05-27T08:38:16.254986+0000","last_clean":"2024-05-27T08:38:16.254986+0000","last_became_active":"2024-05-27T08:35:07.052500+0000","last_became_peered":"2024-05-27T08:35:07.052500+0000","last_unstale":"2024-05-27T08:38:16.254986+0000","last_undegraded":"2024-05-27T08:38:16.254986+0000","last_fullsized":"2024-05-27T08:38:16.254986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:25.093408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295540+0000","last_change":"2024-05-27T08:35:07.044866+0000","last_active":"2024-05-27T08:38:16.295540+0000","last_peered":"2024-05-27T08:38:16.295540+0000","last_clean":"2024-05-27T08:38:16.295540+0000","last_became_active":"2024-05-27T08:35:07.044554+0000","last_became_peered":"2024-05-27T08:35:07.044554+0000","last_unstale":"2024-05-27T08:38:16.295540+0000","last_undegraded":"2024-05-27T08:38:16.295540+0000","last_fullsized":"2024-05-27T08:38:16.295540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:45.624970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438667+0000","last_change":"2024-05-27T08:35:07.043985+0000","last_active":"2024-05-27T08:38:16.438667+0000","last_peered":"2024-05-27T08:38:16.438667+0000","last_clean":"2024-05-27T08:38:16.438667+0000","last_became_active":"2024-05-27T08:35:07.043694+0000","last_became_peered":"2024-05-27T08:35:07.043694+0000","last_unstale":"2024-05-27T08:38:16.438667+0000","last_undegraded":"2024-05-27T08:38:16.438667+0000","last_fullsized":"2024-05-27T08:38:16.438667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:22:49.796670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438460+0000","last_change":"2024-05-27T08:35:07.045542+0000","last_active":"2024-05-27T08:38:16.438460+0000","last_peered":"2024-05-27T08:38:16.438460+0000","last_clean":"2024-05-27T08:38:16.438460+0000","last_became_active":"2024-05-27T08:35:07.045209+0000","last_became_peered":"2024-05-27T08:35:07.045209+0000","last_unstale":"2024-05-27T08:38:16.438460+0000","last_undegraded":"2024-05-27T08:38:16.438460+0000","last_fullsized":"2024-05-27T08:38:16.438460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:57.733089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219687+0000","last_change":"2024-05-27T08:35:07.047193+0000","last_active":"2024-05-27T08:38:16.219687+0000","last_peered":"2024-05-27T08:38:16.219687+0000","last_clean":"2024-05-27T08:38:16.219687+0000","last_became_active":"2024-05-27T08:35:07.046862+0000","last_became_peered":"2024-05-27T08:35:07.046862+0000","last_unstale":"2024-05-27T08:38:16.219687+0000","last_undegraded":"2024-05-27T08:38:16.219687+0000","last_fullsized":"2024-05-27T08:38:16.219687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:21.288642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439576+0000","last_change":"2024-05-27T08:35:07.045452+0000","last_active":"2024-05-27T08:38:16.439576+0000","last_peered":"2024-05-27T08:38:16.439576+0000","last_clean":"2024-05-27T08:38:16.439576+0000","last_became_active":"2024-05-27T08:35:07.044783+0000","last_became_peered":"2024-05-27T08:35:07.044783+0000","last_unstale":"2024-05-27T08:38:16.439576+0000","last_undegraded":"2024-05-27T08:38:16.439576+0000","last_fullsized":"2024-05-27T08:38:16.439576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:03.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317955+0000","last_change":"2024-05-27T08:35:07.042864+0000","last_active":"2024-05-27T08:38:16.317955+0000","last_peered":"2024-05-27T08:38:16.317955+0000","last_clean":"2024-05-27T08:38:16.317955+0000","last_became_active":"2024-05-27T08:35:07.042523+0000","last_became_peered":"2024-05-27T08:35:07.042523+0000","last_unstale":"2024-05-27T08:38:16.317955+0000","last_undegraded":"2024-05-27T08:38:16.317955+0000","last_fullsized":"2024-05-27T08:38:16.317955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:34.275977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438978+0000","last_change":"2024-05-27T08:35:07.042667+0000","last_active":"2024-05-27T08:38:16.438978+0000","last_peered":"2024-05-27T08:38:16.438978+0000","last_clean":"2024-05-27T08:38:16.438978+0000","last_became_active":"2024-05-27T08:35:07.042090+0000","last_became_peered":"2024-05-27T08:35:07.042090+0000","last_unstale":"2024-05-27T08:38:16.438978+0000","last_undegraded":"2024-05-27T08:38:16.438978+0000","last_fullsized":"2024-05-27T08:38:16.438978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:28.764641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":236,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317975+0000","last_change":"2024-05-27T08:35:58.391406+0000","last_active":"2024-05-27T08:38:16.317975+0000","last_peered":"2024-05-27T08:38:16.317975+0000","last_clean":"2024-05-27T08:38:16.317975+0000","last_became_active":"2024-05-27T08:35:58.376035+0000","last_became_peered":"2024-05-27T08:35:58.376035+0000","last_unstale":"2024-05-27T08:38:16.317975+0000","last_undegraded":"2024-05-27T08:38:16.317975+0000","last_fullsized":"2024-05-27T08:38:16.317975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:54.751467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317866+0000","last_change":"2024-05-27T08:35:58.376043+0000","last_active":"2024-05-27T08:38:16.317866+0000","last_peered":"2024-05-27T08:38:16.317866+0000","last_clean":"2024-05-27T08:38:16.317866+0000","last_became_active":"2024-05-27T08:35:58.375695+0000","last_became_peered":"2024-05-27T08:35:58.375695+0000","last_unstale":"2024-05-27T08:38:16.317866+0000","last_undegraded":"2024-05-27T08:38:16.317866+0000","last_fullsized":"2024-05-27T08:38:16.317866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:30:40.875555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219253+0000","last_change":"2024-05-27T08:35:04.245013+0000","last_active":"2024-05-27T08:38:16.219253+0000","last_peered":"2024-05-27T08:38:16.219253+0000","last_clean":"2024-05-27T08:38:16.219253+0000","last_became_active":"2024-05-27T08:35:04.244629+0000","last_became_peered":"2024-05-27T08:35:04.244629+0000","last_unstale":"2024-05-27T08:38:16.219253+0000","last_undegraded":"2024-05-27T08:38:16.219253+0000","last_fullsized":"2024-05-27T08:38:16.219253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:41:43.747592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439484+0000","last_change":"2024-05-27T08:35:07.041776+0000","last_active":"2024-05-27T08:38:16.439484+0000","last_peered":"2024-05-27T08:38:16.439484+0000","last_clean":"2024-05-27T08:38:16.439484+0000","last_became_active":"2024-05-27T08:35:07.041400+0000","last_became_peered":"2024-05-27T08:35:07.041400+0000","last_unstale":"2024-05-27T08:38:16.439484+0000","last_undegraded":"2024-05-27T08:38:16.439484+0000","last_fullsized":"2024-05-27T08:38:16.439484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:10.091310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320491+0000","last_change":"2024-05-27T08:35:04.023072+0000","last_active":"2024-05-27T08:38:16.320491+0000","last_peered":"2024-05-27T08:38:16.320491+0000","last_clean":"2024-05-27T08:38:16.320491+0000","last_became_active":"2024-05-27T08:35:04.022232+0000","last_became_peered":"2024-05-27T08:35:04.022232+0000","last_unstale":"2024-05-27T08:38:16.320491+0000","last_undegraded":"2024-05-27T08:38:16.320491+0000","last_fullsized":"2024-05-27T08:38:16.320491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:49:57.240621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219808+0000","last_change":"2024-05-27T08:35:07.044727+0000","last_active":"2024-05-27T08:38:16.219808+0000","last_peered":"2024-05-27T08:38:16.219808+0000","last_clean":"2024-05-27T08:38:16.219808+0000","last_became_active":"2024-05-27T08:35:07.044352+0000","last_became_peered":"2024-05-27T08:35:07.044352+0000","last_unstale":"2024-05-27T08:38:16.219808+0000","last_undegraded":"2024-05-27T08:38:16.219808+0000","last_fullsized":"2024-05-27T08:38:16.219808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:52:43.473606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439227+0000","last_change":"2024-05-27T08:35:58.382377+0000","last_active":"2024-05-27T08:38:16.439227+0000","last_peered":"2024-05-27T08:38:16.439227+0000","last_clean":"2024-05-27T08:38:16.439227+0000","last_became_active":"2024-05-27T08:35:58.376093+0000","last_became_peered":"2024-05-27T08:35:58.376093+0000","last_unstale":"2024-05-27T08:38:16.439227+0000","last_undegraded":"2024-05-27T08:38:16.439227+0000","last_fullsized":"2024-05-27T08:38:16.439227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:33.792015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217745+0000","last_change":"2024-05-27T08:35:07.055562+0000","last_active":"2024-05-27T08:38:16.217745+0000","last_peered":"2024-05-27T08:38:16.217745+0000","last_clean":"2024-05-27T08:38:16.217745+0000","last_became_active":"2024-05-27T08:35:07.055209+0000","last_became_peered":"2024-05-27T08:35:07.055209+0000","last_unstale":"2024-05-27T08:38:16.217745+0000","last_undegraded":"2024-05-27T08:38:16.217745+0000","last_fullsized":"2024-05-27T08:38:16.217745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:04.920534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":214,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.520944+0000","last_change":"2024-05-27T08:35:04.022778+0000","last_active":"2024-05-27T08:38:16.520944+0000","last_peered":"2024-05-27T08:38:16.520944+0000","last_clean":"2024-05-27T08:38:16.520944+0000","last_became_active":"2024-05-27T08:35:04.022311+0000","last_became_peered":"2024-05-27T08:35:04.022311+0000","last_unstale":"2024-05-27T08:38:16.520944+0000","last_undegraded":"2024-05-27T08:38:16.520944+0000","last_fullsized":"2024-05-27T08:38:16.520944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:48.174774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320107+0000","last_change":"2024-05-27T08:35:07.040182+0000","last_active":"2024-05-27T08:38:16.320107+0000","last_peered":"2024-05-27T08:38:16.320107+0000","last_clean":"2024-05-27T08:38:16.320107+0000","last_became_active":"2024-05-27T08:35:07.039800+0000","last_became_peered":"2024-05-27T08:35:07.039800+0000","last_unstale":"2024-05-27T08:38:16.320107+0000","last_undegraded":"2024-05-27T08:38:16.320107+0000","last_fullsized":"2024-05-27T08:38:16.320107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:53:53.470284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325736+0000","last_change":"2024-05-27T08:35:04.023757+0000","last_active":"2024-05-27T08:38:16.325736+0000","last_peered":"2024-05-27T08:38:16.325736+0000","last_clean":"2024-05-27T08:38:16.325736+0000","last_became_active":"2024-05-27T08:35:04.023428+0000","last_became_peered":"2024-05-27T08:35:04.023428+0000","last_unstale":"2024-05-27T08:38:16.325736+0000","last_undegraded":"2024-05-27T08:38:16.325736+0000","last_fullsized":"2024-05-27T08:38:16.325736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:45.669006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317688+0000","last_change":"2024-05-27T08:35:58.407697+0000","last_active":"2024-05-27T08:38:16.317688+0000","last_peered":"2024-05-27T08:38:16.317688+0000","last_clean":"2024-05-27T08:38:16.317688+0000","last_became_active":"2024-05-27T08:35:58.407379+0000","last_became_peered":"2024-05-27T08:35:58.407379+0000","last_unstale":"2024-05-27T08:38:16.317688+0000","last_undegraded":"2024-05-27T08:38:16.317688+0000","last_fullsized":"2024-05-27T08:38:16.317688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:06.534914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295392+0000","last_change":"2024-05-27T08:35:04.015581+0000","last_active":"2024-05-27T08:38:16.295392+0000","last_peered":"2024-05-27T08:38:16.295392+0000","last_clean":"2024-05-27T08:38:16.295392+0000","last_became_active":"2024-05-27T08:35:04.015215+0000","last_became_peered":"2024-05-27T08:35:04.015215+0000","last_unstale":"2024-05-27T08:38:16.295392+0000","last_undegraded":"2024-05-27T08:38:16.295392+0000","last_fullsized":"2024-05-27T08:38:16.295392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:21:51.942008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218704+0000","last_change":"2024-05-27T08:35:07.049798+0000","last_active":"2024-05-27T08:38:16.218704+0000","last_peered":"2024-05-27T08:38:16.218704+0000","last_clean":"2024-05-27T08:38:16.218704+0000","last_became_active":"2024-05-27T08:35:07.048686+0000","last_became_peered":"2024-05-27T08:35:07.048686+0000","last_unstale":"2024-05-27T08:38:16.218704+0000","last_undegraded":"2024-05-27T08:38:16.218704+0000","last_fullsized":"2024-05-27T08:38:16.218704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:39:23.817581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217470+0000","last_change":"2024-05-27T08:35:04.023826+0000","last_active":"2024-05-27T08:38:16.217470+0000","last_peered":"2024-05-27T08:38:16.217470+0000","last_clean":"2024-05-27T08:38:16.217470+0000","last_became_active":"2024-05-27T08:35:04.023410+0000","last_became_peered":"2024-05-27T08:35:04.023410+0000","last_unstale":"2024-05-27T08:38:16.217470+0000","last_undegraded":"2024-05-27T08:38:16.217470+0000","last_fullsized":"2024-05-27T08:38:16.217470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:38:47.855186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317617+0000","last_change":"2024-05-27T08:35:07.038810+0000","last_active":"2024-05-27T08:38:16.317617+0000","last_peered":"2024-05-27T08:38:16.317617+0000","last_clean":"2024-05-27T08:38:16.317617+0000","last_became_active":"2024-05-27T08:35:07.038480+0000","last_became_peered":"2024-05-27T08:35:07.038480+0000","last_unstale":"2024-05-27T08:38:16.317617+0000","last_undegraded":"2024-05-27T08:38:16.317617+0000","last_fullsized":"2024-05-27T08:38:16.317617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:14.429081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":382,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.519928+0000","last_change":"2024-05-27T08:35:04.026934+0000","last_active":"2024-05-27T08:38:16.519928+0000","last_peered":"2024-05-27T08:38:16.519928+0000","last_clean":"2024-05-27T08:38:16.519928+0000","last_became_active":"2024-05-27T08:35:04.026577+0000","last_became_peered":"2024-05-27T08:35:04.026577+0000","last_unstale":"2024-05-27T08:38:16.519928+0000","last_undegraded":"2024-05-27T08:38:16.519928+0000","last_fullsized":"2024-05-27T08:38:16.519928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:59:56.067856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":346,"num_read_kb":346,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.220241+0000","last_change":"2024-05-27T08:35:07.043840+0000","last_active":"2024-05-27T08:38:16.220241+0000","last_peered":"2024-05-27T08:38:16.220241+0000","last_clean":"2024-05-27T08:38:16.220241+0000","last_became_active":"2024-05-27T08:35:07.043224+0000","last_became_peered":"2024-05-27T08:35:07.043224+0000","last_unstale":"2024-05-27T08:38:16.220241+0000","last_undegraded":"2024-05-27T08:38:16.220241+0000","last_fullsized":"2024-05-27T08:38:16.220241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:20.212038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438880+0000","last_change":"2024-05-27T08:35:04.022602+0000","last_active":"2024-05-27T08:38:16.438880+0000","last_peered":"2024-05-27T08:38:16.438880+0000","last_clean":"2024-05-27T08:38:16.438880+0000","last_became_active":"2024-05-27T08:35:04.022203+0000","last_became_peered":"2024-05-27T08:35:04.022203+0000","last_unstale":"2024-05-27T08:38:16.438880+0000","last_undegraded":"2024-05-27T08:38:16.438880+0000","last_fullsized":"2024-05-27T08:38:16.438880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:51.490696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439428+0000","last_change":"2024-05-27T08:35:07.047257+0000","last_active":"2024-05-27T08:38:16.439428+0000","last_peered":"2024-05-27T08:38:16.439428+0000","last_clean":"2024-05-27T08:38:16.439428+0000","last_became_active":"2024-05-27T08:35:07.046639+0000","last_became_peered":"2024-05-27T08:35:07.046639+0000","last_unstale":"2024-05-27T08:38:16.439428+0000","last_undegraded":"2024-05-27T08:38:16.439428+0000","last_fullsized":"2024-05-27T08:38:16.439428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:04.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217915+0000","last_change":"2024-05-27T08:35:04.245295+0000","last_active":"2024-05-27T08:38:16.217915+0000","last_peered":"2024-05-27T08:38:16.217915+0000","last_clean":"2024-05-27T08:38:16.217915+0000","last_became_active":"2024-05-27T08:35:04.244927+0000","last_became_peered":"2024-05-27T08:35:04.244927+0000","last_unstale":"2024-05-27T08:38:16.217915+0000","last_undegraded":"2024-05-27T08:38:16.217915+0000","last_fullsized":"2024-05-27T08:38:16.217915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:58.893480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219915+0000","last_change":"2024-05-27T08:35:07.048872+0000","last_active":"2024-05-27T08:38:16.219915+0000","last_peered":"2024-05-27T08:38:16.219915+0000","last_clean":"2024-05-27T08:38:16.219915+0000","last_became_active":"2024-05-27T08:35:07.048488+0000","last_became_peered":"2024-05-27T08:35:07.048488+0000","last_unstale":"2024-05-27T08:38:16.219915+0000","last_undegraded":"2024-05-27T08:38:16.219915+0000","last_fullsized":"2024-05-27T08:38:16.219915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:31.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":427,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.520427+0000","last_change":"2024-05-27T08:35:04.024718+0000","last_active":"2024-05-27T08:38:16.520427+0000","last_peered":"2024-05-27T08:38:16.520427+0000","last_clean":"2024-05-27T08:38:16.520427+0000","last_became_active":"2024-05-27T08:35:04.024155+0000","last_became_peered":"2024-05-27T08:35:04.024155+0000","last_unstale":"2024-05-27T08:38:16.520427+0000","last_undegraded":"2024-05-27T08:38:16.520427+0000","last_fullsized":"2024-05-27T08:38:16.520427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:22.573337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":359,"num_read_kb":244,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320317+0000","last_change":"2024-05-27T08:35:07.051342+0000","last_active":"2024-05-27T08:38:16.320317+0000","last_peered":"2024-05-27T08:38:16.320317+0000","last_clean":"2024-05-27T08:38:16.320317+0000","last_became_active":"2024-05-27T08:35:07.051030+0000","last_became_peered":"2024-05-27T08:35:07.051030+0000","last_unstale":"2024-05-27T08:38:16.320317+0000","last_undegraded":"2024-05-27T08:38:16.320317+0000","last_fullsized":"2024-05-27T08:38:16.320317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:48.245036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438727+0000","last_change":"2024-05-27T08:35:04.023717+0000","last_active":"2024-05-27T08:38:16.438727+0000","last_peered":"2024-05-27T08:38:16.438727+0000","last_clean":"2024-05-27T08:38:16.438727+0000","last_became_active":"2024-05-27T08:35:04.022017+0000","last_became_peered":"2024-05-27T08:35:04.022017+0000","last_unstale":"2024-05-27T08:38:16.438727+0000","last_undegraded":"2024-05-27T08:38:16.438727+0000","last_fullsized":"2024-05-27T08:38:16.438727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:55.834307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295399+0000","last_change":"2024-05-27T08:35:07.043980+0000","last_active":"2024-05-27T08:38:16.295399+0000","last_peered":"2024-05-27T08:38:16.295399+0000","last_clean":"2024-05-27T08:38:16.295399+0000","last_became_active":"2024-05-27T08:35:07.043672+0000","last_became_peered":"2024-05-27T08:35:07.043672+0000","last_unstale":"2024-05-27T08:38:16.295399+0000","last_undegraded":"2024-05-27T08:38:16.295399+0000","last_fullsized":"2024-05-27T08:38:16.295399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:03:30.243966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217554+0000","last_change":"2024-05-27T08:35:04.022731+0000","last_active":"2024-05-27T08:38:16.217554+0000","last_peered":"2024-05-27T08:38:16.217554+0000","last_clean":"2024-05-27T08:38:16.217554+0000","last_became_active":"2024-05-27T08:35:04.022349+0000","last_became_peered":"2024-05-27T08:35:04.022349+0000","last_unstale":"2024-05-27T08:38:16.217554+0000","last_undegraded":"2024-05-27T08:38:16.217554+0000","last_fullsized":"2024-05-27T08:38:16.217554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:38.651520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438589+0000","last_change":"2024-05-27T08:35:07.052623+0000","last_active":"2024-05-27T08:38:16.438589+0000","last_peered":"2024-05-27T08:38:16.438589+0000","last_clean":"2024-05-27T08:38:16.438589+0000","last_became_active":"2024-05-27T08:35:07.052313+0000","last_became_peered":"2024-05-27T08:35:07.052313+0000","last_unstale":"2024-05-27T08:38:16.438589+0000","last_undegraded":"2024-05-27T08:38:16.438589+0000","last_fullsized":"2024-05-27T08:38:16.438589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:24.612523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":214,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.536811+0000","last_change":"2024-05-27T08:35:04.023756+0000","last_active":"2024-05-27T08:38:16.536811+0000","last_peered":"2024-05-27T08:38:16.536811+0000","last_clean":"2024-05-27T08:38:16.536811+0000","last_became_active":"2024-05-27T08:35:04.023401+0000","last_became_peered":"2024-05-27T08:35:04.023401+0000","last_unstale":"2024-05-27T08:38:16.536811+0000","last_undegraded":"2024-05-27T08:38:16.536811+0000","last_fullsized":"2024-05-27T08:38:16.536811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:34.887246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":179,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438391+0000","last_change":"2024-05-27T08:35:07.050720+0000","last_active":"2024-05-27T08:38:16.438391+0000","last_peered":"2024-05-27T08:38:16.438391+0000","last_clean":"2024-05-27T08:38:16.438391+0000","last_became_active":"2024-05-27T08:35:07.050135+0000","last_became_peered":"2024-05-27T08:35:07.050135+0000","last_unstale":"2024-05-27T08:38:16.438391+0000","last_undegraded":"2024-05-27T08:38:16.438391+0000","last_fullsized":"2024-05-27T08:38:16.438391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:40.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320069+0000","last_change":"2024-05-27T08:35:58.421002+0000","last_active":"2024-05-27T08:38:16.320069+0000","last_peered":"2024-05-27T08:38:16.320069+0000","last_clean":"2024-05-27T08:38:16.320069+0000","last_became_active":"2024-05-27T08:35:58.408889+0000","last_became_peered":"2024-05-27T08:35:58.408889+0000","last_unstale":"2024-05-27T08:38:16.320069+0000","last_undegraded":"2024-05-27T08:38:16.320069+0000","last_fullsized":"2024-05-27T08:38:16.320069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:29.705436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217489+0000","last_change":"2024-05-27T08:35:07.050504+0000","last_active":"2024-05-27T08:38:16.217489+0000","last_peered":"2024-05-27T08:38:16.217489+0000","last_clean":"2024-05-27T08:38:16.217489+0000","last_became_active":"2024-05-27T08:35:07.049368+0000","last_became_peered":"2024-05-27T08:35:07.049368+0000","last_unstale":"2024-05-27T08:38:16.217489+0000","last_undegraded":"2024-05-27T08:38:16.217489+0000","last_fullsized":"2024-05-27T08:38:16.217489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:52.996313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320461+0000","last_change":"2024-05-27T08:35:04.023924+0000","last_active":"2024-05-27T08:38:16.320461+0000","last_peered":"2024-05-27T08:38:16.320461+0000","last_clean":"2024-05-27T08:38:16.320461+0000","last_became_active":"2024-05-27T08:35:04.023548+0000","last_became_peered":"2024-05-27T08:35:04.023548+0000","last_unstale":"2024-05-27T08:38:16.320461+0000","last_undegraded":"2024-05-27T08:38:16.320461+0000","last_fullsized":"2024-05-27T08:38:16.320461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:50.441490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439437+0000","last_change":"2024-05-27T08:35:58.376335+0000","last_active":"2024-05-27T08:38:16.439437+0000","last_peered":"2024-05-27T08:38:16.439437+0000","last_clean":"2024-05-27T08:38:16.439437+0000","last_became_active":"2024-05-27T08:35:58.375926+0000","last_became_peered":"2024-05-27T08:35:58.375926+0000","last_unstale":"2024-05-27T08:38:16.439437+0000","last_undegraded":"2024-05-27T08:38:16.439437+0000","last_fullsized":"2024-05-27T08:38:16.439437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:33.892172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317833+0000","last_change":"2024-05-27T08:35:58.374310+0000","last_active":"2024-05-27T08:38:16.317833+0000","last_peered":"2024-05-27T08:38:16.317833+0000","last_clean":"2024-05-27T08:38:16.317833+0000","last_became_active":"2024-05-27T08:35:58.373929+0000","last_became_peered":"2024-05-27T08:35:58.373929+0000","last_unstale":"2024-05-27T08:38:16.317833+0000","last_undegraded":"2024-05-27T08:38:16.317833+0000","last_fullsized":"2024-05-27T08:38:16.317833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:20.901125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325887+0000","last_change":"2024-05-27T08:35:07.039182+0000","last_active":"2024-05-27T08:38:16.325887+0000","last_peered":"2024-05-27T08:38:16.325887+0000","last_clean":"2024-05-27T08:38:16.325887+0000","last_became_active":"2024-05-27T08:35:07.038793+0000","last_became_peered":"2024-05-27T08:35:07.038793+0000","last_unstale":"2024-05-27T08:38:16.325887+0000","last_undegraded":"2024-05-27T08:38:16.325887+0000","last_fullsized":"2024-05-27T08:38:16.325887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:03.839681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'7","reported_seq":211,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.552162+0000","last_change":"2024-05-27T08:35:04.023194+0000","last_active":"2024-05-27T08:38:16.552162+0000","last_peered":"2024-05-27T08:38:16.552162+0000","last_clean":"2024-05-27T08:38:16.552162+0000","last_became_active":"2024-05-27T08:35:04.022823+0000","last_became_peered":"2024-05-27T08:35:04.022823+0000","last_unstale":"2024-05-27T08:38:16.552162+0000","last_undegraded":"2024-05-27T08:38:16.552162+0000","last_fullsized":"2024-05-27T08:38:16.552162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:44:28.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325773+0000","last_change":"2024-05-27T08:35:07.050717+0000","last_active":"2024-05-27T08:38:16.325773+0000","last_peered":"2024-05-27T08:38:16.325773+0000","last_clean":"2024-05-27T08:38:16.325773+0000","last_became_active":"2024-05-27T08:35:07.050413+0000","last_became_peered":"2024-05-27T08:35:07.050413+0000","last_unstale":"2024-05-27T08:38:16.325773+0000","last_undegraded":"2024-05-27T08:38:16.325773+0000","last_fullsized":"2024-05-27T08:38:16.325773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:00:27.693141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217821+0000","last_change":"2024-05-27T08:35:07.053614+0000","last_active":"2024-05-27T08:38:16.217821+0000","last_peered":"2024-05-27T08:38:16.217821+0000","last_clean":"2024-05-27T08:38:16.217821+0000","last_became_active":"2024-05-27T08:35:07.053253+0000","last_became_peered":"2024-05-27T08:35:07.053253+0000","last_unstale":"2024-05-27T08:38:16.217821+0000","last_undegraded":"2024-05-27T08:38:16.217821+0000","last_fullsized":"2024-05-27T08:38:16.217821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:15:08.985984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":210,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.463537+0000","last_change":"2024-05-27T08:35:04.246500+0000","last_active":"2024-05-27T08:38:16.463537+0000","last_peered":"2024-05-27T08:38:16.463537+0000","last_clean":"2024-05-27T08:38:16.463537+0000","last_became_active":"2024-05-27T08:35:04.244575+0000","last_became_peered":"2024-05-27T08:35:04.244575+0000","last_unstale":"2024-05-27T08:38:16.463537+0000","last_undegraded":"2024-05-27T08:38:16.463537+0000","last_fullsized":"2024-05-27T08:38:16.463537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:24:10.818057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218987+0000","last_change":"2024-05-27T08:35:07.048488+0000","last_active":"2024-05-27T08:38:16.218987+0000","last_peered":"2024-05-27T08:38:16.218987+0000","last_clean":"2024-05-27T08:38:16.218987+0000","last_became_active":"2024-05-27T08:35:07.048172+0000","last_became_peered":"2024-05-27T08:35:07.048172+0000","last_unstale":"2024-05-27T08:38:16.218987+0000","last_undegraded":"2024-05-27T08:38:16.218987+0000","last_fullsized":"2024-05-27T08:38:16.218987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:01.632944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255015+0000","last_change":"2024-05-27T08:35:04.022807+0000","last_active":"2024-05-27T08:38:16.255015+0000","last_peered":"2024-05-27T08:38:16.255015+0000","last_clean":"2024-05-27T08:38:16.255015+0000","last_became_active":"2024-05-27T08:35:04.021198+0000","last_became_peered":"2024-05-27T08:35:04.021198+0000","last_unstale":"2024-05-27T08:38:16.255015+0000","last_undegraded":"2024-05-27T08:38:16.255015+0000","last_fullsized":"2024-05-27T08:38:16.255015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:42:10.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":103,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438484+0000","last_change":"2024-05-27T08:33:45.683600+0000","last_active":"2024-05-27T08:38:16.438484+0000","last_peered":"2024-05-27T08:38:16.438484+0000","last_clean":"2024-05-27T08:38:16.438484+0000","last_became_active":"2024-05-27T08:33:45.677989+0000","last_became_peered":"2024-05-27T08:33:45.677989+0000","last_unstale":"2024-05-27T08:38:16.438484+0000","last_undegraded":"2024-05-27T08:38:16.438484+0000","last_fullsized":"2024-05-27T08:38:16.438484+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_clean_scrub_stamp":"2024-05-27T08:30:25.507727+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:48:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295270+0000","last_change":"2024-05-27T08:35:07.045879+0000","last_active":"2024-05-27T08:38:16.295270+0000","last_peered":"2024-05-27T08:38:16.295270+0000","last_clean":"2024-05-27T08:38:16.295270+0000","last_became_active":"2024-05-27T08:35:07.045522+0000","last_became_peered":"2024-05-27T08:35:07.045522+0000","last_unstale":"2024-05-27T08:38:16.295270+0000","last_undegraded":"2024-05-27T08:38:16.295270+0000","last_fullsized":"2024-05-27T08:38:16.295270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:30.393769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438619+0000","last_change":"2024-05-27T08:35:04.022419+0000","last_active":"2024-05-27T08:38:16.438619+0000","last_peered":"2024-05-27T08:38:16.438619+0000","last_clean":"2024-05-27T08:38:16.438619+0000","last_became_active":"2024-05-27T08:35:04.022039+0000","last_became_peered":"2024-05-27T08:35:04.022039+0000","last_unstale":"2024-05-27T08:38:16.438619+0000","last_undegraded":"2024-05-27T08:38:16.438619+0000","last_fullsized":"2024-05-27T08:38:16.438619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:31.260596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320374+0000","last_change":"2024-05-27T08:35:04.245214+0000","last_active":"2024-05-27T08:38:16.320374+0000","last_peered":"2024-05-27T08:38:16.320374+0000","last_clean":"2024-05-27T08:38:16.320374+0000","last_became_active":"2024-05-27T08:35:04.244768+0000","last_became_peered":"2024-05-27T08:35:04.244768+0000","last_unstale":"2024-05-27T08:38:16.320374+0000","last_undegraded":"2024-05-27T08:38:16.320374+0000","last_fullsized":"2024-05-27T08:38:16.320374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:34.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295585+0000","last_change":"2024-05-27T08:35:07.042721+0000","last_active":"2024-05-27T08:38:16.295585+0000","last_peered":"2024-05-27T08:38:16.295585+0000","last_clean":"2024-05-27T08:38:16.295585+0000","last_became_active":"2024-05-27T08:35:07.042435+0000","last_became_peered":"2024-05-27T08:35:07.042435+0000","last_unstale":"2024-05-27T08:38:16.295585+0000","last_undegraded":"2024-05-27T08:38:16.295585+0000","last_fullsized":"2024-05-27T08:38:16.295585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:49.186753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'52","reported_seq":103,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.424706+0000","last_change":"2024-05-27T08:35:04.017995+0000","last_active":"2024-05-27T08:38:16.424706+0000","last_peered":"2024-05-27T08:38:16.424706+0000","last_clean":"2024-05-27T08:38:16.424706+0000","last_became_active":"2024-05-27T08:35:04.017675+0000","last_became_peered":"2024-05-27T08:35:04.017675+0000","last_unstale":"2024-05-27T08:38:16.424706+0000","last_undegraded":"2024-05-27T08:38:16.424706+0000","last_fullsized":"2024-05-27T08:38:16.424706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:23:15.452624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219103+0000","last_change":"2024-05-27T08:35:07.046553+0000","last_active":"2024-05-27T08:38:16.219103+0000","last_peered":"2024-05-27T08:38:16.219103+0000","last_clean":"2024-05-27T08:38:16.219103+0000","last_became_active":"2024-05-27T08:35:07.046229+0000","last_became_peered":"2024-05-27T08:35:07.046229+0000","last_unstale":"2024-05-27T08:38:16.219103+0000","last_undegraded":"2024-05-27T08:38:16.219103+0000","last_fullsized":"2024-05-27T08:38:16.219103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:21.024307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255144+0000","last_change":"2024-05-27T08:35:04.262439+0000","last_active":"2024-05-27T08:38:16.255144+0000","last_peered":"2024-05-27T08:38:16.255144+0000","last_clean":"2024-05-27T08:38:16.255144+0000","last_became_active":"2024-05-27T08:35:04.261958+0000","last_became_peered":"2024-05-27T08:35:04.261958+0000","last_unstale":"2024-05-27T08:38:16.255144+0000","last_undegraded":"2024-05-27T08:38:16.255144+0000","last_fullsized":"2024-05-27T08:38:16.255144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:11:05.763068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439536+0000","last_change":"2024-05-27T08:35:07.042307+0000","last_active":"2024-05-27T08:38:16.439536+0000","last_peered":"2024-05-27T08:38:16.439536+0000","last_clean":"2024-05-27T08:38:16.439536+0000","last_became_active":"2024-05-27T08:35:07.042001+0000","last_became_peered":"2024-05-27T08:35:07.042001+0000","last_unstale":"2024-05-27T08:38:16.439536+0000","last_undegraded":"2024-05-27T08:38:16.439536+0000","last_fullsized":"2024-05-27T08:38:16.439536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:51.030157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219698+0000","last_change":"2024-05-27T08:35:04.245436+0000","last_active":"2024-05-27T08:38:16.219698+0000","last_peered":"2024-05-27T08:38:16.219698+0000","last_clean":"2024-05-27T08:38:16.219698+0000","last_became_active":"2024-05-27T08:35:04.245039+0000","last_became_peered":"2024-05-27T08:35:04.245039+0000","last_unstale":"2024-05-27T08:38:16.219698+0000","last_undegraded":"2024-05-27T08:38:16.219698+0000","last_fullsized":"2024-05-27T08:38:16.219698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:10:11.756864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320069+0000","last_change":"2024-05-27T08:35:07.051524+0000","last_active":"2024-05-27T08:38:16.320069+0000","last_peered":"2024-05-27T08:38:16.320069+0000","last_clean":"2024-05-27T08:38:16.320069+0000","last_became_active":"2024-05-27T08:35:07.050676+0000","last_became_peered":"2024-05-27T08:35:07.050676+0000","last_unstale":"2024-05-27T08:38:16.320069+0000","last_undegraded":"2024-05-27T08:38:16.320069+0000","last_fullsized":"2024-05-27T08:38:16.320069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:06.818977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218795+0000","last_change":"2024-05-27T08:35:04.022064+0000","last_active":"2024-05-27T08:38:16.218795+0000","last_peered":"2024-05-27T08:38:16.218795+0000","last_clean":"2024-05-27T08:38:16.218795+0000","last_became_active":"2024-05-27T08:35:04.021691+0000","last_became_peered":"2024-05-27T08:35:04.021691+0000","last_unstale":"2024-05-27T08:38:16.218795+0000","last_undegraded":"2024-05-27T08:38:16.218795+0000","last_fullsized":"2024-05-27T08:38:16.218795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:54.557756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439157+0000","last_change":"2024-05-27T08:35:07.049666+0000","last_active":"2024-05-27T08:38:16.439157+0000","last_peered":"2024-05-27T08:38:16.439157+0000","last_clean":"2024-05-27T08:38:16.439157+0000","last_became_active":"2024-05-27T08:35:07.049312+0000","last_became_peered":"2024-05-27T08:35:07.049312+0000","last_unstale":"2024-05-27T08:38:16.439157+0000","last_undegraded":"2024-05-27T08:38:16.439157+0000","last_fullsized":"2024-05-27T08:38:16.439157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:37:53.704582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438548+0000","last_change":"2024-05-27T08:35:04.018457+0000","last_active":"2024-05-27T08:38:16.438548+0000","last_peered":"2024-05-27T08:38:16.438548+0000","last_clean":"2024-05-27T08:38:16.438548+0000","last_became_active":"2024-05-27T08:35:04.018008+0000","last_became_peered":"2024-05-27T08:35:04.018008+0000","last_unstale":"2024-05-27T08:38:16.438548+0000","last_undegraded":"2024-05-27T08:38:16.438548+0000","last_fullsized":"2024-05-27T08:38:16.438548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:08:13.228725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219211+0000","last_change":"2024-05-27T08:35:07.047040+0000","last_active":"2024-05-27T08:38:16.219211+0000","last_peered":"2024-05-27T08:38:16.219211+0000","last_clean":"2024-05-27T08:38:16.219211+0000","last_became_active":"2024-05-27T08:35:07.046731+0000","last_became_peered":"2024-05-27T08:35:07.046731+0000","last_unstale":"2024-05-27T08:38:16.219211+0000","last_undegraded":"2024-05-27T08:38:16.219211+0000","last_fullsized":"2024-05-27T08:38:16.219211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:25.368349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320164+0000","last_change":"2024-05-27T08:35:04.022804+0000","last_active":"2024-05-27T08:38:16.320164+0000","last_peered":"2024-05-27T08:38:16.320164+0000","last_clean":"2024-05-27T08:38:16.320164+0000","last_became_active":"2024-05-27T08:35:04.022458+0000","last_became_peered":"2024-05-27T08:35:04.022458+0000","last_unstale":"2024-05-27T08:38:16.320164+0000","last_undegraded":"2024-05-27T08:38:16.320164+0000","last_fullsized":"2024-05-27T08:38:16.320164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:01.029226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438657+0000","last_change":"2024-05-27T08:35:07.049136+0000","last_active":"2024-05-27T08:38:16.438657+0000","last_peered":"2024-05-27T08:38:16.438657+0000","last_clean":"2024-05-27T08:38:16.438657+0000","last_became_active":"2024-05-27T08:35:07.048793+0000","last_became_peered":"2024-05-27T08:35:07.048793+0000","last_unstale":"2024-05-27T08:38:16.438657+0000","last_undegraded":"2024-05-27T08:38:16.438657+0000","last_fullsized":"2024-05-27T08:38:16.438657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:10:21.191743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":210,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.520790+0000","last_change":"2024-05-27T08:35:04.262102+0000","last_active":"2024-05-27T08:38:16.520790+0000","last_peered":"2024-05-27T08:38:16.520790+0000","last_clean":"2024-05-27T08:38:16.520790+0000","last_became_active":"2024-05-27T08:35:04.261665+0000","last_became_peered":"2024-05-27T08:35:04.261665+0000","last_unstale":"2024-05-27T08:38:16.520790+0000","last_undegraded":"2024-05-27T08:38:16.520790+0000","last_fullsized":"2024-05-27T08:38:16.520790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:57:57.712197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438962+0000","last_change":"2024-05-27T08:35:07.051557+0000","last_active":"2024-05-27T08:38:16.438962+0000","last_peered":"2024-05-27T08:38:16.438962+0000","last_clean":"2024-05-27T08:38:16.438962+0000","last_became_active":"2024-05-27T08:35:07.051246+0000","last_became_peered":"2024-05-27T08:35:07.051246+0000","last_unstale":"2024-05-27T08:38:16.438962+0000","last_undegraded":"2024-05-27T08:38:16.438962+0000","last_fullsized":"2024-05-27T08:38:16.438962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:08:13.248301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.422467+0000","last_change":"2024-05-27T08:35:04.022328+0000","last_active":"2024-05-27T08:38:16.422467+0000","last_peered":"2024-05-27T08:38:16.422467+0000","last_clean":"2024-05-27T08:38:16.422467+0000","last_became_active":"2024-05-27T08:35:04.021711+0000","last_became_peered":"2024-05-27T08:35:04.021711+0000","last_unstale":"2024-05-27T08:38:16.422467+0000","last_undegraded":"2024-05-27T08:38:16.422467+0000","last_fullsized":"2024-05-27T08:38:16.422467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:32.841474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438669+0000","last_change":"2024-05-27T08:35:07.056768+0000","last_active":"2024-05-27T08:38:16.438669+0000","last_peered":"2024-05-27T08:38:16.438669+0000","last_clean":"2024-05-27T08:38:16.438669+0000","last_became_active":"2024-05-27T08:35:07.056482+0000","last_became_peered":"2024-05-27T08:35:07.056482+0000","last_unstale":"2024-05-27T08:38:16.438669+0000","last_undegraded":"2024-05-27T08:38:16.438669+0000","last_fullsized":"2024-05-27T08:38:16.438669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:23.515625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295576+0000","last_change":"2024-05-27T08:35:04.016613+0000","last_active":"2024-05-27T08:38:16.295576+0000","last_peered":"2024-05-27T08:38:16.295576+0000","last_clean":"2024-05-27T08:38:16.295576+0000","last_became_active":"2024-05-27T08:35:04.016268+0000","last_became_peered":"2024-05-27T08:35:04.016268+0000","last_unstale":"2024-05-27T08:38:16.295576+0000","last_undegraded":"2024-05-27T08:38:16.295576+0000","last_fullsized":"2024-05-27T08:38:16.295576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:11.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438981+0000","last_change":"2024-05-27T08:35:07.051515+0000","last_active":"2024-05-27T08:38:16.438981+0000","last_peered":"2024-05-27T08:38:16.438981+0000","last_clean":"2024-05-27T08:38:16.438981+0000","last_became_active":"2024-05-27T08:35:07.051197+0000","last_became_peered":"2024-05-27T08:35:07.051197+0000","last_unstale":"2024-05-27T08:38:16.438981+0000","last_undegraded":"2024-05-27T08:38:16.438981+0000","last_fullsized":"2024-05-27T08:38:16.438981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:49.233474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439171+0000","last_change":"2024-05-27T08:35:07.051960+0000","last_active":"2024-05-27T08:38:16.439171+0000","last_peered":"2024-05-27T08:38:16.439171+0000","last_clean":"2024-05-27T08:38:16.439171+0000","last_became_active":"2024-05-27T08:35:07.051395+0000","last_became_peered":"2024-05-27T08:35:07.051395+0000","last_unstale":"2024-05-27T08:38:16.439171+0000","last_undegraded":"2024-05-27T08:38:16.439171+0000","last_fullsized":"2024-05-27T08:38:16.439171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:57.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438517+0000","last_change":"2024-05-27T08:35:07.050096+0000","last_active":"2024-05-27T08:38:16.438517+0000","last_peered":"2024-05-27T08:38:16.438517+0000","last_clean":"2024-05-27T08:38:16.438517+0000","last_became_active":"2024-05-27T08:35:07.049755+0000","last_became_peered":"2024-05-27T08:35:07.049755+0000","last_unstale":"2024-05-27T08:38:16.438517+0000","last_undegraded":"2024-05-27T08:38:16.438517+0000","last_fullsized":"2024-05-27T08:38:16.438517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:39.768128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438481+0000","last_change":"2024-05-27T08:35:07.048281+0000","last_active":"2024-05-27T08:38:16.438481+0000","last_peered":"2024-05-27T08:38:16.438481+0000","last_clean":"2024-05-27T08:38:16.438481+0000","last_became_active":"2024-05-27T08:35:07.047879+0000","last_became_peered":"2024-05-27T08:35:07.047879+0000","last_unstale":"2024-05-27T08:38:16.438481+0000","last_undegraded":"2024-05-27T08:38:16.438481+0000","last_fullsized":"2024-05-27T08:38:16.438481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:43.631671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438821+0000","last_change":"2024-05-27T08:35:07.050501+0000","last_active":"2024-05-27T08:38:16.438821+0000","last_peered":"2024-05-27T08:38:16.438821+0000","last_clean":"2024-05-27T08:38:16.438821+0000","last_became_active":"2024-05-27T08:35:07.050159+0000","last_became_peered":"2024-05-27T08:35:07.050159+0000","last_unstale":"2024-05-27T08:38:16.438821+0000","last_undegraded":"2024-05-27T08:38:16.438821+0000","last_fullsized":"2024-05-27T08:38:16.438821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:17:23.293019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295514+0000","last_change":"2024-05-27T08:35:58.372446+0000","last_active":"2024-05-27T08:38:16.295514+0000","last_peered":"2024-05-27T08:38:16.295514+0000","last_clean":"2024-05-27T08:38:16.295514+0000","last_became_active":"2024-05-27T08:35:58.372080+0000","last_became_peered":"2024-05-27T08:35:58.372080+0000","last_unstale":"2024-05-27T08:38:16.295514+0000","last_undegraded":"2024-05-27T08:38:16.295514+0000","last_fullsized":"2024-05-27T08:38:16.295514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:57.828720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439444+0000","last_change":"2024-05-27T08:35:07.052382+0000","last_active":"2024-05-27T08:38:16.439444+0000","last_peered":"2024-05-27T08:38:16.439444+0000","last_clean":"2024-05-27T08:38:16.439444+0000","last_became_active":"2024-05-27T08:35:07.052049+0000","last_became_peered":"2024-05-27T08:35:07.052049+0000","last_unstale":"2024-05-27T08:38:16.439444+0000","last_undegraded":"2024-05-27T08:38:16.439444+0000","last_fullsized":"2024-05-27T08:38:16.439444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:08.912872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438897+0000","last_change":"2024-05-27T08:35:07.043810+0000","last_active":"2024-05-27T08:38:16.438897+0000","last_peered":"2024-05-27T08:38:16.438897+0000","last_clean":"2024-05-27T08:38:16.438897+0000","last_became_active":"2024-05-27T08:35:07.043449+0000","last_became_peered":"2024-05-27T08:35:07.043449+0000","last_unstale":"2024-05-27T08:38:16.438897+0000","last_undegraded":"2024-05-27T08:38:16.438897+0000","last_fullsized":"2024-05-27T08:38:16.438897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:18.089405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218995+0000","last_change":"2024-05-27T08:35:58.371407+0000","last_active":"2024-05-27T08:38:16.218995+0000","last_peered":"2024-05-27T08:38:16.218995+0000","last_clean":"2024-05-27T08:38:16.218995+0000","last_became_active":"2024-05-27T08:35:58.370963+0000","last_became_peered":"2024-05-27T08:35:58.370963+0000","last_unstale":"2024-05-27T08:38:16.218995+0000","last_undegraded":"2024-05-27T08:38:16.218995+0000","last_fullsized":"2024-05-27T08:38:16.218995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:23.458573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217613+0000","last_change":"2024-05-27T08:35:07.049960+0000","last_active":"2024-05-27T08:38:16.217613+0000","last_peered":"2024-05-27T08:38:16.217613+0000","last_clean":"2024-05-27T08:38:16.217613+0000","last_became_active":"2024-05-27T08:35:07.049595+0000","last_became_peered":"2024-05-27T08:35:07.049595+0000","last_unstale":"2024-05-27T08:38:16.217613+0000","last_undegraded":"2024-05-27T08:38:16.217613+0000","last_fullsized":"2024-05-27T08:38:16.217613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:18.182658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218743+0000","last_change":"2024-05-27T08:35:07.043790+0000","last_active":"2024-05-27T08:38:16.218743+0000","last_peered":"2024-05-27T08:38:16.218743+0000","last_clean":"2024-05-27T08:38:16.218743+0000","last_became_active":"2024-05-27T08:35:07.043476+0000","last_became_peered":"2024-05-27T08:35:07.043476+0000","last_unstale":"2024-05-27T08:38:16.218743+0000","last_undegraded":"2024-05-27T08:38:16.218743+0000","last_fullsized":"2024-05-27T08:38:16.218743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:01.626979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219848+0000","last_change":"2024-05-27T08:35:07.044928+0000","last_active":"2024-05-27T08:38:16.219848+0000","last_peered":"2024-05-27T08:38:16.219848+0000","last_clean":"2024-05-27T08:38:16.219848+0000","last_became_active":"2024-05-27T08:35:07.044577+0000","last_became_peered":"2024-05-27T08:35:07.044577+0000","last_unstale":"2024-05-27T08:38:16.219848+0000","last_undegraded":"2024-05-27T08:38:16.219848+0000","last_fullsized":"2024-05-27T08:38:16.219848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:36.560525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295715+0000","last_change":"2024-05-27T08:35:07.045067+0000","last_active":"2024-05-27T08:38:16.295715+0000","last_peered":"2024-05-27T08:38:16.295715+0000","last_clean":"2024-05-27T08:38:16.295715+0000","last_became_active":"2024-05-27T08:35:07.044759+0000","last_became_peered":"2024-05-27T08:35:07.044759+0000","last_unstale":"2024-05-27T08:38:16.295715+0000","last_undegraded":"2024-05-27T08:38:16.295715+0000","last_fullsized":"2024-05-27T08:38:16.295715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:08.440135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439347+0000","last_change":"2024-05-27T08:35:07.048186+0000","last_active":"2024-05-27T08:38:16.439347+0000","last_peered":"2024-05-27T08:38:16.439347+0000","last_clean":"2024-05-27T08:38:16.439347+0000","last_became_active":"2024-05-27T08:35:07.047828+0000","last_became_peered":"2024-05-27T08:35:07.047828+0000","last_unstale":"2024-05-27T08:38:16.439347+0000","last_undegraded":"2024-05-27T08:38:16.439347+0000","last_fullsized":"2024-05-27T08:38:16.439347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:09.928234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600648,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1867,"num_read_kb":1047,"num_write":379,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1801944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":458064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1654784,"data_stored":1114344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939711,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32580,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712572,"statfs":{"total":95995035648,"available":95961673728,"internally_reserved":0,"allocated":1368064,"data_stored":850237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135940,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28340,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716812,"statfs":{"total":95995035648,"available":95966015488,"internally_reserved":0,"allocated":1351680,"data_stored":849361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332168,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1041942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495694,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":958464,"data_stored":457437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":525496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":458468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":525547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117084,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32644,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712508,"statfs":{"total":95995035648,"available":95961608192,"internally_reserved":0,"allocated":1503232,"data_stored":981379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920855,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":457634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691924,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1716224,"data_stored":1181727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T08:38:21.630 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T08:38:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi077 to 176.1M 2024-05-27T08:38:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Unable to set osd_memory_target on smithi077 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-27T08:38:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi077:/etc/ceph/ceph.conf 2024-05-27T08:38:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi092:/etc/ceph/ceph.conf 2024-05-27T08:38:21.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi179:/etc/ceph/ceph.conf 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.conf 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi092:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: Updating smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.737 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:21.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:21 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:38:21.916 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:23.078 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:22 smithi077 ceph-mon[32007]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:23.078 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:22 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.078 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:22 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.078 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:22 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:38:23.078 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:22 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:38:23.078 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:22 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:22 smithi179 ceph-mon[36481]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:22 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:22 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:22 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:38:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:22 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:38:23.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:22 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:22 smithi092 ceph-mon[36049]: from='client.24902 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:22 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:22 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:22 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:38:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:22 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:38:23.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:22 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:38:23.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 -- 172.21.15.77:0/3152331208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434108520 msgr2=0x7ff434108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:23.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/3152331208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434108520 0x7ff434108900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff424009990 tx=0x7ff424009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 -- 172.21.15.77:0/3152331208 shutdown_connections 2024-05-27T08:38:23.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/3152331208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff434102e00 0x7ff43410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/3152331208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434108520 0x7ff434108900 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.518 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/3152331208 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff434102440 0x7ff4341028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 -- 172.21.15.77:0/3152331208 >> 172.21.15.77:0/3152331208 conn(0x7ff4340793f0 msgr2=0x7ff434079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 -- 172.21.15.77:0/3152331208 shutdown_connections 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 -- 172.21.15.77:0/3152331208 wait complete. 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 Processor -- start 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.484+0000 7ff43a4ce700 1 -- start start 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff43a4ce700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434102440 0x7ff43419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff43a4ce700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff434102e00 0x7ff43419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff43a4ce700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff434108520 0x7ff434198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff43a4ce700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341125f0 con 0x7ff434102440 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff43a4ce700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff434112470 con 0x7ff434108520 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff43a4ce700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff434112770 con 0x7ff434102e00 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434102440 0x7ff43419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:23.519 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434102440 0x7ff43419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39794/0 (socket says 172.21.15.77:39794) 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 -- 172.21.15.77:0/1364908122 learned_addr learned my addr 172.21.15.77:0/1364908122 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff438a6b700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff434108520 0x7ff434198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 -- 172.21.15.77:0/1364908122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff434102e00 msgr2=0x7ff43419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff4337fe700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff434102e00 0x7ff43419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff434102e00 0x7ff43419ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 -- 172.21.15.77:0/1364908122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff434108520 msgr2=0x7ff434198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff434108520 0x7ff434198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 -- 172.21.15.77:0/1364908122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff434199070 con 0x7ff434102440 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff438a6b700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff434108520 0x7ff434198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.485+0000 7ff433fff700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434102440 0x7ff43419e750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff424000c00 tx=0x7ff4240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.486+0000 7ff4317fa700 1 -- 172.21.15.77:0/1364908122 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424027070 con 0x7ff434102440 2024-05-27T08:38:23.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.486+0000 7ff4317fa700 1 -- 172.21.15.77:0/1364908122 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff424003fa0 con 0x7ff434102440 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.486+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff434199300 con 0x7ff434102440 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.486+0000 7ff4317fa700 1 -- 172.21.15.77:0/1364908122 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424022db0 con 0x7ff434102440 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.486+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4341a5620 con 0x7ff434102440 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.487+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43410a9b0 con 0x7ff434102440 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.488+0000 7ff4317fa700 1 -- 172.21.15.77:0/1364908122 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7ff42402a070 con 0x7ff434102440 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.489+0000 7ff4317fa700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff41c0b2b00 0x7ff41c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.489+0000 7ff4337fe700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff41c0b2b00 0x7ff41c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.489+0000 7ff4317fa700 1 -- 172.21.15.77:0/1364908122 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff4240e3420 con 0x7ff434102440 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.489+0000 7ff4337fe700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff41c0b2b00 0x7ff41c0b4fc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff434079aa0 tx=0x7ff428005cf0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:23.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.493+0000 7ff4317fa700 1 -- 172.21.15.77:0/1364908122 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff424013070 con 0x7ff434102440 2024-05-27T08:38:23.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.695+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff434063980 con 0x7ff41c0b2b00 2024-05-27T08:38:23.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.703+0000 7ff4317fa700 1 -- 172.21.15.77:0/1364908122 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257813 (secure 0 0 0) 0x7ff434063980 con 0x7ff41c0b2b00 2024-05-27T08:38:23.709 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:38:23.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff41c0b2b00 msgr2=0x7ff41c0b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:23.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff41c0b2b00 0x7ff41c0b4fc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff434079aa0 tx=0x7ff428005cf0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434102440 msgr2=0x7ff43419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434102440 0x7ff43419e750 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff424000c00 tx=0x7ff4240047b0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 shutdown_connections 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff41c0b2b00 0x7ff41c0b4fc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff434108520 0x7ff434198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff434102440 0x7ff43419e750 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 --2- 172.21.15.77:0/1364908122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff434102e00 0x7ff43419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 >> 172.21.15.77:0/1364908122 conn(0x7ff4340793f0 msgr2=0x7ff434101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 shutdown_connections 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:23.706+0000 7ff43a4ce700 1 -- 172.21.15.77:0/1364908122 wait complete. 2024-05-27T08:38:23.711 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-05-27T08:38:24.077 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:23 smithi077 ceph-mon[32007]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:24.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:23 smithi179 ceph-mon[36481]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:24.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:23 smithi092 ceph-mon[36049]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:38:24.308 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-27T08:38:22.232501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191250,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1948,"num_read_kb":1116,"num_write":504,"num_write_kb":3620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2953206,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":900,"ondisk_log_size":900,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":372612,"kb_used_data":14920,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124569212,"statfs":{"total":1151940427776,"available":1151558873088,"internally_reserved":0,"allocated":15278080,"data_stored":9098268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.990535"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439549+0000","last_change":"2024-05-27T08:35:07.046361+0000","last_active":"2024-05-27T08:38:16.439549+0000","last_peered":"2024-05-27T08:38:16.439549+0000","last_clean":"2024-05-27T08:38:16.439549+0000","last_became_active":"2024-05-27T08:35:07.045954+0000","last_became_peered":"2024-05-27T08:35:07.045954+0000","last_unstale":"2024-05-27T08:38:16.439549+0000","last_undegraded":"2024-05-27T08:38:16.439549+0000","last_fullsized":"2024-05-27T08:38:16.439549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:03:47.458964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439255+0000","last_change":"2024-05-27T08:35:07.043725+0000","last_active":"2024-05-27T08:38:16.439255+0000","last_peered":"2024-05-27T08:38:16.439255+0000","last_clean":"2024-05-27T08:38:16.439255+0000","last_became_active":"2024-05-27T08:35:07.043447+0000","last_became_peered":"2024-05-27T08:35:07.043447+0000","last_unstale":"2024-05-27T08:38:16.439255+0000","last_undegraded":"2024-05-27T08:38:16.439255+0000","last_fullsized":"2024-05-27T08:38:16.439255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:44.509202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439075+0000","last_change":"2024-05-27T08:35:07.048075+0000","last_active":"2024-05-27T08:38:16.439075+0000","last_peered":"2024-05-27T08:38:16.439075+0000","last_clean":"2024-05-27T08:38:16.439075+0000","last_became_active":"2024-05-27T08:35:07.047687+0000","last_became_peered":"2024-05-27T08:35:07.047687+0000","last_unstale":"2024-05-27T08:38:16.439075+0000","last_undegraded":"2024-05-27T08:38:16.439075+0000","last_fullsized":"2024-05-27T08:38:16.439075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:05:15.197231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255117+0000","last_change":"2024-05-27T08:35:07.046809+0000","last_active":"2024-05-27T08:38:16.255117+0000","last_peered":"2024-05-27T08:38:16.255117+0000","last_clean":"2024-05-27T08:38:16.255117+0000","last_became_active":"2024-05-27T08:35:07.046453+0000","last_became_peered":"2024-05-27T08:35:07.046453+0000","last_unstale":"2024-05-27T08:38:16.255117+0000","last_undegraded":"2024-05-27T08:38:16.255117+0000","last_fullsized":"2024-05-27T08:38:16.255117+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:43.209613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320175+0000","last_change":"2024-05-27T08:35:07.040242+0000","last_active":"2024-05-27T08:38:16.320175+0000","last_peered":"2024-05-27T08:38:16.320175+0000","last_clean":"2024-05-27T08:38:16.320175+0000","last_became_active":"2024-05-27T08:35:07.039860+0000","last_became_peered":"2024-05-27T08:35:07.039860+0000","last_unstale":"2024-05-27T08:38:16.320175+0000","last_undegraded":"2024-05-27T08:38:16.320175+0000","last_fullsized":"2024-05-27T08:38:16.320175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:49.212799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325893+0000","last_change":"2024-05-27T08:35:07.048176+0000","last_active":"2024-05-27T08:38:16.325893+0000","last_peered":"2024-05-27T08:38:16.325893+0000","last_clean":"2024-05-27T08:38:16.325893+0000","last_became_active":"2024-05-27T08:35:07.047860+0000","last_became_peered":"2024-05-27T08:35:07.047860+0000","last_unstale":"2024-05-27T08:38:16.325893+0000","last_undegraded":"2024-05-27T08:38:16.325893+0000","last_fullsized":"2024-05-27T08:38:16.325893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:19.459704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438445+0000","last_change":"2024-05-27T08:35:07.052285+0000","last_active":"2024-05-27T08:38:16.438445+0000","last_peered":"2024-05-27T08:38:16.438445+0000","last_clean":"2024-05-27T08:38:16.438445+0000","last_became_active":"2024-05-27T08:35:07.051970+0000","last_became_peered":"2024-05-27T08:35:07.051970+0000","last_unstale":"2024-05-27T08:38:16.438445+0000","last_undegraded":"2024-05-27T08:38:16.438445+0000","last_fullsized":"2024-05-27T08:38:16.438445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:56.543022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218781+0000","last_change":"2024-05-27T08:35:07.047527+0000","last_active":"2024-05-27T08:38:16.218781+0000","last_peered":"2024-05-27T08:38:16.218781+0000","last_clean":"2024-05-27T08:38:16.218781+0000","last_became_active":"2024-05-27T08:35:07.047204+0000","last_became_peered":"2024-05-27T08:35:07.047204+0000","last_unstale":"2024-05-27T08:38:16.218781+0000","last_undegraded":"2024-05-27T08:38:16.218781+0000","last_fullsized":"2024-05-27T08:38:16.218781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:04:43.547847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.220295+0000","last_change":"2024-05-27T08:35:07.049108+0000","last_active":"2024-05-27T08:38:16.220295+0000","last_peered":"2024-05-27T08:38:16.220295+0000","last_clean":"2024-05-27T08:38:16.220295+0000","last_became_active":"2024-05-27T08:35:07.048540+0000","last_became_peered":"2024-05-27T08:35:07.048540+0000","last_unstale":"2024-05-27T08:38:16.220295+0000","last_undegraded":"2024-05-27T08:38:16.220295+0000","last_fullsized":"2024-05-27T08:38:16.220295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:56.219362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.326127+0000","last_change":"2024-05-27T08:35:07.049775+0000","last_active":"2024-05-27T08:38:16.326127+0000","last_peered":"2024-05-27T08:38:16.326127+0000","last_clean":"2024-05-27T08:38:16.326127+0000","last_became_active":"2024-05-27T08:35:07.049466+0000","last_became_peered":"2024-05-27T08:35:07.049466+0000","last_unstale":"2024-05-27T08:38:16.326127+0000","last_undegraded":"2024-05-27T08:38:16.326127+0000","last_fullsized":"2024-05-27T08:38:16.326127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:18:30.394390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438881+0000","last_change":"2024-05-27T08:35:07.050870+0000","last_active":"2024-05-27T08:38:16.438881+0000","last_peered":"2024-05-27T08:38:16.438881+0000","last_clean":"2024-05-27T08:38:16.438881+0000","last_became_active":"2024-05-27T08:35:07.050527+0000","last_became_peered":"2024-05-27T08:35:07.050527+0000","last_unstale":"2024-05-27T08:38:16.438881+0000","last_undegraded":"2024-05-27T08:38:16.438881+0000","last_fullsized":"2024-05-27T08:38:16.438881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:56:41.477053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.254986+0000","last_change":"2024-05-27T08:35:07.052955+0000","last_active":"2024-05-27T08:38:16.254986+0000","last_peered":"2024-05-27T08:38:16.254986+0000","last_clean":"2024-05-27T08:38:16.254986+0000","last_became_active":"2024-05-27T08:35:07.052500+0000","last_became_peered":"2024-05-27T08:35:07.052500+0000","last_unstale":"2024-05-27T08:38:16.254986+0000","last_undegraded":"2024-05-27T08:38:16.254986+0000","last_fullsized":"2024-05-27T08:38:16.254986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:25.093408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295540+0000","last_change":"2024-05-27T08:35:07.044866+0000","last_active":"2024-05-27T08:38:16.295540+0000","last_peered":"2024-05-27T08:38:16.295540+0000","last_clean":"2024-05-27T08:38:16.295540+0000","last_became_active":"2024-05-27T08:35:07.044554+0000","last_became_peered":"2024-05-27T08:35:07.044554+0000","last_unstale":"2024-05-27T08:38:16.295540+0000","last_undegraded":"2024-05-27T08:38:16.295540+0000","last_fullsized":"2024-05-27T08:38:16.295540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:45.624970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438667+0000","last_change":"2024-05-27T08:35:07.043985+0000","last_active":"2024-05-27T08:38:16.438667+0000","last_peered":"2024-05-27T08:38:16.438667+0000","last_clean":"2024-05-27T08:38:16.438667+0000","last_became_active":"2024-05-27T08:35:07.043694+0000","last_became_peered":"2024-05-27T08:35:07.043694+0000","last_unstale":"2024-05-27T08:38:16.438667+0000","last_undegraded":"2024-05-27T08:38:16.438667+0000","last_fullsized":"2024-05-27T08:38:16.438667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:22:49.796670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438460+0000","last_change":"2024-05-27T08:35:07.045542+0000","last_active":"2024-05-27T08:38:16.438460+0000","last_peered":"2024-05-27T08:38:16.438460+0000","last_clean":"2024-05-27T08:38:16.438460+0000","last_became_active":"2024-05-27T08:35:07.045209+0000","last_became_peered":"2024-05-27T08:35:07.045209+0000","last_unstale":"2024-05-27T08:38:16.438460+0000","last_undegraded":"2024-05-27T08:38:16.438460+0000","last_fullsized":"2024-05-27T08:38:16.438460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:57.733089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219687+0000","last_change":"2024-05-27T08:35:07.047193+0000","last_active":"2024-05-27T08:38:16.219687+0000","last_peered":"2024-05-27T08:38:16.219687+0000","last_clean":"2024-05-27T08:38:16.219687+0000","last_became_active":"2024-05-27T08:35:07.046862+0000","last_became_peered":"2024-05-27T08:35:07.046862+0000","last_unstale":"2024-05-27T08:38:16.219687+0000","last_undegraded":"2024-05-27T08:38:16.219687+0000","last_fullsized":"2024-05-27T08:38:16.219687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:21.288642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439576+0000","last_change":"2024-05-27T08:35:07.045452+0000","last_active":"2024-05-27T08:38:16.439576+0000","last_peered":"2024-05-27T08:38:16.439576+0000","last_clean":"2024-05-27T08:38:16.439576+0000","last_became_active":"2024-05-27T08:35:07.044783+0000","last_became_peered":"2024-05-27T08:35:07.044783+0000","last_unstale":"2024-05-27T08:38:16.439576+0000","last_undegraded":"2024-05-27T08:38:16.439576+0000","last_fullsized":"2024-05-27T08:38:16.439576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:03.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317955+0000","last_change":"2024-05-27T08:35:07.042864+0000","last_active":"2024-05-27T08:38:16.317955+0000","last_peered":"2024-05-27T08:38:16.317955+0000","last_clean":"2024-05-27T08:38:16.317955+0000","last_became_active":"2024-05-27T08:35:07.042523+0000","last_became_peered":"2024-05-27T08:35:07.042523+0000","last_unstale":"2024-05-27T08:38:16.317955+0000","last_undegraded":"2024-05-27T08:38:16.317955+0000","last_fullsized":"2024-05-27T08:38:16.317955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:34.275977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438978+0000","last_change":"2024-05-27T08:35:07.042667+0000","last_active":"2024-05-27T08:38:16.438978+0000","last_peered":"2024-05-27T08:38:16.438978+0000","last_clean":"2024-05-27T08:38:16.438978+0000","last_became_active":"2024-05-27T08:35:07.042090+0000","last_became_peered":"2024-05-27T08:35:07.042090+0000","last_unstale":"2024-05-27T08:38:16.438978+0000","last_undegraded":"2024-05-27T08:38:16.438978+0000","last_fullsized":"2024-05-27T08:38:16.438978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:28.764641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":236,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317975+0000","last_change":"2024-05-27T08:35:58.391406+0000","last_active":"2024-05-27T08:38:16.317975+0000","last_peered":"2024-05-27T08:38:16.317975+0000","last_clean":"2024-05-27T08:38:16.317975+0000","last_became_active":"2024-05-27T08:35:58.376035+0000","last_became_peered":"2024-05-27T08:35:58.376035+0000","last_unstale":"2024-05-27T08:38:16.317975+0000","last_undegraded":"2024-05-27T08:38:16.317975+0000","last_fullsized":"2024-05-27T08:38:16.317975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:54.751467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":168,"num_read_kb":168,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317866+0000","last_change":"2024-05-27T08:35:58.376043+0000","last_active":"2024-05-27T08:38:16.317866+0000","last_peered":"2024-05-27T08:38:16.317866+0000","last_clean":"2024-05-27T08:38:16.317866+0000","last_became_active":"2024-05-27T08:35:58.375695+0000","last_became_peered":"2024-05-27T08:35:58.375695+0000","last_unstale":"2024-05-27T08:38:16.317866+0000","last_undegraded":"2024-05-27T08:38:16.317866+0000","last_fullsized":"2024-05-27T08:38:16.317866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:30:40.875555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219253+0000","last_change":"2024-05-27T08:35:04.245013+0000","last_active":"2024-05-27T08:38:16.219253+0000","last_peered":"2024-05-27T08:38:16.219253+0000","last_clean":"2024-05-27T08:38:16.219253+0000","last_became_active":"2024-05-27T08:35:04.244629+0000","last_became_peered":"2024-05-27T08:35:04.244629+0000","last_unstale":"2024-05-27T08:38:16.219253+0000","last_undegraded":"2024-05-27T08:38:16.219253+0000","last_fullsized":"2024-05-27T08:38:16.219253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:41:43.747592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439484+0000","last_change":"2024-05-27T08:35:07.041776+0000","last_active":"2024-05-27T08:38:16.439484+0000","last_peered":"2024-05-27T08:38:16.439484+0000","last_clean":"2024-05-27T08:38:16.439484+0000","last_became_active":"2024-05-27T08:35:07.041400+0000","last_became_peered":"2024-05-27T08:35:07.041400+0000","last_unstale":"2024-05-27T08:38:16.439484+0000","last_undegraded":"2024-05-27T08:38:16.439484+0000","last_fullsized":"2024-05-27T08:38:16.439484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:10.091310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320491+0000","last_change":"2024-05-27T08:35:04.023072+0000","last_active":"2024-05-27T08:38:16.320491+0000","last_peered":"2024-05-27T08:38:16.320491+0000","last_clean":"2024-05-27T08:38:16.320491+0000","last_became_active":"2024-05-27T08:35:04.022232+0000","last_became_peered":"2024-05-27T08:35:04.022232+0000","last_unstale":"2024-05-27T08:38:16.320491+0000","last_undegraded":"2024-05-27T08:38:16.320491+0000","last_fullsized":"2024-05-27T08:38:16.320491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:49:57.240621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219808+0000","last_change":"2024-05-27T08:35:07.044727+0000","last_active":"2024-05-27T08:38:16.219808+0000","last_peered":"2024-05-27T08:38:16.219808+0000","last_clean":"2024-05-27T08:38:16.219808+0000","last_became_active":"2024-05-27T08:35:07.044352+0000","last_became_peered":"2024-05-27T08:35:07.044352+0000","last_unstale":"2024-05-27T08:38:16.219808+0000","last_undegraded":"2024-05-27T08:38:16.219808+0000","last_fullsized":"2024-05-27T08:38:16.219808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:52:43.473606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439227+0000","last_change":"2024-05-27T08:35:58.382377+0000","last_active":"2024-05-27T08:38:16.439227+0000","last_peered":"2024-05-27T08:38:16.439227+0000","last_clean":"2024-05-27T08:38:16.439227+0000","last_became_active":"2024-05-27T08:35:58.376093+0000","last_became_peered":"2024-05-27T08:35:58.376093+0000","last_unstale":"2024-05-27T08:38:16.439227+0000","last_undegraded":"2024-05-27T08:38:16.439227+0000","last_fullsized":"2024-05-27T08:38:16.439227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:33.792015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217745+0000","last_change":"2024-05-27T08:35:07.055562+0000","last_active":"2024-05-27T08:38:16.217745+0000","last_peered":"2024-05-27T08:38:16.217745+0000","last_clean":"2024-05-27T08:38:16.217745+0000","last_became_active":"2024-05-27T08:35:07.055209+0000","last_became_peered":"2024-05-27T08:35:07.055209+0000","last_unstale":"2024-05-27T08:38:16.217745+0000","last_undegraded":"2024-05-27T08:38:16.217745+0000","last_fullsized":"2024-05-27T08:38:16.217745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:04.920534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":214,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.520944+0000","last_change":"2024-05-27T08:35:04.022778+0000","last_active":"2024-05-27T08:38:16.520944+0000","last_peered":"2024-05-27T08:38:16.520944+0000","last_clean":"2024-05-27T08:38:16.520944+0000","last_became_active":"2024-05-27T08:35:04.022311+0000","last_became_peered":"2024-05-27T08:35:04.022311+0000","last_unstale":"2024-05-27T08:38:16.520944+0000","last_undegraded":"2024-05-27T08:38:16.520944+0000","last_fullsized":"2024-05-27T08:38:16.520944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:48.174774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320107+0000","last_change":"2024-05-27T08:35:07.040182+0000","last_active":"2024-05-27T08:38:16.320107+0000","last_peered":"2024-05-27T08:38:16.320107+0000","last_clean":"2024-05-27T08:38:16.320107+0000","last_became_active":"2024-05-27T08:35:07.039800+0000","last_became_peered":"2024-05-27T08:35:07.039800+0000","last_unstale":"2024-05-27T08:38:16.320107+0000","last_undegraded":"2024-05-27T08:38:16.320107+0000","last_fullsized":"2024-05-27T08:38:16.320107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:53:53.470284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325736+0000","last_change":"2024-05-27T08:35:04.023757+0000","last_active":"2024-05-27T08:38:16.325736+0000","last_peered":"2024-05-27T08:38:16.325736+0000","last_clean":"2024-05-27T08:38:16.325736+0000","last_became_active":"2024-05-27T08:35:04.023428+0000","last_became_peered":"2024-05-27T08:35:04.023428+0000","last_unstale":"2024-05-27T08:38:16.325736+0000","last_undegraded":"2024-05-27T08:38:16.325736+0000","last_fullsized":"2024-05-27T08:38:16.325736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:45.669006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317688+0000","last_change":"2024-05-27T08:35:58.407697+0000","last_active":"2024-05-27T08:38:16.317688+0000","last_peered":"2024-05-27T08:38:16.317688+0000","last_clean":"2024-05-27T08:38:16.317688+0000","last_became_active":"2024-05-27T08:35:58.407379+0000","last_became_peered":"2024-05-27T08:35:58.407379+0000","last_unstale":"2024-05-27T08:38:16.317688+0000","last_undegraded":"2024-05-27T08:38:16.317688+0000","last_fullsized":"2024-05-27T08:38:16.317688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:06.534914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295392+0000","last_change":"2024-05-27T08:35:04.015581+0000","last_active":"2024-05-27T08:38:16.295392+0000","last_peered":"2024-05-27T08:38:16.295392+0000","last_clean":"2024-05-27T08:38:16.295392+0000","last_became_active":"2024-05-27T08:35:04.015215+0000","last_became_peered":"2024-05-27T08:35:04.015215+0000","last_unstale":"2024-05-27T08:38:16.295392+0000","last_undegraded":"2024-05-27T08:38:16.295392+0000","last_fullsized":"2024-05-27T08:38:16.295392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:21:51.942008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218704+0000","last_change":"2024-05-27T08:35:07.049798+0000","last_active":"2024-05-27T08:38:16.218704+0000","last_peered":"2024-05-27T08:38:16.218704+0000","last_clean":"2024-05-27T08:38:16.218704+0000","last_became_active":"2024-05-27T08:35:07.048686+0000","last_became_peered":"2024-05-27T08:35:07.048686+0000","last_unstale":"2024-05-27T08:38:16.218704+0000","last_undegraded":"2024-05-27T08:38:16.218704+0000","last_fullsized":"2024-05-27T08:38:16.218704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:39:23.817581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217470+0000","last_change":"2024-05-27T08:35:04.023826+0000","last_active":"2024-05-27T08:38:16.217470+0000","last_peered":"2024-05-27T08:38:16.217470+0000","last_clean":"2024-05-27T08:38:16.217470+0000","last_became_active":"2024-05-27T08:35:04.023410+0000","last_became_peered":"2024-05-27T08:35:04.023410+0000","last_unstale":"2024-05-27T08:38:16.217470+0000","last_undegraded":"2024-05-27T08:38:16.217470+0000","last_fullsized":"2024-05-27T08:38:16.217470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:38:47.855186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317617+0000","last_change":"2024-05-27T08:35:07.038810+0000","last_active":"2024-05-27T08:38:16.317617+0000","last_peered":"2024-05-27T08:38:16.317617+0000","last_clean":"2024-05-27T08:38:16.317617+0000","last_became_active":"2024-05-27T08:35:07.038480+0000","last_became_peered":"2024-05-27T08:35:07.038480+0000","last_unstale":"2024-05-27T08:38:16.317617+0000","last_undegraded":"2024-05-27T08:38:16.317617+0000","last_fullsized":"2024-05-27T08:38:16.317617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:14.429081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":382,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.519928+0000","last_change":"2024-05-27T08:35:04.026934+0000","last_active":"2024-05-27T08:38:16.519928+0000","last_peered":"2024-05-27T08:38:16.519928+0000","last_clean":"2024-05-27T08:38:16.519928+0000","last_became_active":"2024-05-27T08:35:04.026577+0000","last_became_peered":"2024-05-27T08:35:04.026577+0000","last_unstale":"2024-05-27T08:38:16.519928+0000","last_undegraded":"2024-05-27T08:38:16.519928+0000","last_fullsized":"2024-05-27T08:38:16.519928+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:59:56.067856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":346,"num_read_kb":346,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.220241+0000","last_change":"2024-05-27T08:35:07.043840+0000","last_active":"2024-05-27T08:38:16.220241+0000","last_peered":"2024-05-27T08:38:16.220241+0000","last_clean":"2024-05-27T08:38:16.220241+0000","last_became_active":"2024-05-27T08:35:07.043224+0000","last_became_peered":"2024-05-27T08:35:07.043224+0000","last_unstale":"2024-05-27T08:38:16.220241+0000","last_undegraded":"2024-05-27T08:38:16.220241+0000","last_fullsized":"2024-05-27T08:38:16.220241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:20.212038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438880+0000","last_change":"2024-05-27T08:35:04.022602+0000","last_active":"2024-05-27T08:38:16.438880+0000","last_peered":"2024-05-27T08:38:16.438880+0000","last_clean":"2024-05-27T08:38:16.438880+0000","last_became_active":"2024-05-27T08:35:04.022203+0000","last_became_peered":"2024-05-27T08:35:04.022203+0000","last_unstale":"2024-05-27T08:38:16.438880+0000","last_undegraded":"2024-05-27T08:38:16.438880+0000","last_fullsized":"2024-05-27T08:38:16.438880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:51.490696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439428+0000","last_change":"2024-05-27T08:35:07.047257+0000","last_active":"2024-05-27T08:38:16.439428+0000","last_peered":"2024-05-27T08:38:16.439428+0000","last_clean":"2024-05-27T08:38:16.439428+0000","last_became_active":"2024-05-27T08:35:07.046639+0000","last_became_peered":"2024-05-27T08:35:07.046639+0000","last_unstale":"2024-05-27T08:38:16.439428+0000","last_undegraded":"2024-05-27T08:38:16.439428+0000","last_fullsized":"2024-05-27T08:38:16.439428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:04.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217915+0000","last_change":"2024-05-27T08:35:04.245295+0000","last_active":"2024-05-27T08:38:16.217915+0000","last_peered":"2024-05-27T08:38:16.217915+0000","last_clean":"2024-05-27T08:38:16.217915+0000","last_became_active":"2024-05-27T08:35:04.244927+0000","last_became_peered":"2024-05-27T08:35:04.244927+0000","last_unstale":"2024-05-27T08:38:16.217915+0000","last_undegraded":"2024-05-27T08:38:16.217915+0000","last_fullsized":"2024-05-27T08:38:16.217915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:58.893480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219915+0000","last_change":"2024-05-27T08:35:07.048872+0000","last_active":"2024-05-27T08:38:16.219915+0000","last_peered":"2024-05-27T08:38:16.219915+0000","last_clean":"2024-05-27T08:38:16.219915+0000","last_became_active":"2024-05-27T08:35:07.048488+0000","last_became_peered":"2024-05-27T08:35:07.048488+0000","last_unstale":"2024-05-27T08:38:16.219915+0000","last_undegraded":"2024-05-27T08:38:16.219915+0000","last_fullsized":"2024-05-27T08:38:16.219915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:31.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":427,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.520427+0000","last_change":"2024-05-27T08:35:04.024718+0000","last_active":"2024-05-27T08:38:16.520427+0000","last_peered":"2024-05-27T08:38:16.520427+0000","last_clean":"2024-05-27T08:38:16.520427+0000","last_became_active":"2024-05-27T08:35:04.024155+0000","last_became_peered":"2024-05-27T08:35:04.024155+0000","last_unstale":"2024-05-27T08:38:16.520427+0000","last_undegraded":"2024-05-27T08:38:16.520427+0000","last_fullsized":"2024-05-27T08:38:16.520427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:22.573337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":359,"num_read_kb":244,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320317+0000","last_change":"2024-05-27T08:35:07.051342+0000","last_active":"2024-05-27T08:38:16.320317+0000","last_peered":"2024-05-27T08:38:16.320317+0000","last_clean":"2024-05-27T08:38:16.320317+0000","last_became_active":"2024-05-27T08:35:07.051030+0000","last_became_peered":"2024-05-27T08:35:07.051030+0000","last_unstale":"2024-05-27T08:38:16.320317+0000","last_undegraded":"2024-05-27T08:38:16.320317+0000","last_fullsized":"2024-05-27T08:38:16.320317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:48.245036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438727+0000","last_change":"2024-05-27T08:35:04.023717+0000","last_active":"2024-05-27T08:38:16.438727+0000","last_peered":"2024-05-27T08:38:16.438727+0000","last_clean":"2024-05-27T08:38:16.438727+0000","last_became_active":"2024-05-27T08:35:04.022017+0000","last_became_peered":"2024-05-27T08:35:04.022017+0000","last_unstale":"2024-05-27T08:38:16.438727+0000","last_undegraded":"2024-05-27T08:38:16.438727+0000","last_fullsized":"2024-05-27T08:38:16.438727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:55.834307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295399+0000","last_change":"2024-05-27T08:35:07.043980+0000","last_active":"2024-05-27T08:38:16.295399+0000","last_peered":"2024-05-27T08:38:16.295399+0000","last_clean":"2024-05-27T08:38:16.295399+0000","last_became_active":"2024-05-27T08:35:07.043672+0000","last_became_peered":"2024-05-27T08:35:07.043672+0000","last_unstale":"2024-05-27T08:38:16.295399+0000","last_undegraded":"2024-05-27T08:38:16.295399+0000","last_fullsized":"2024-05-27T08:38:16.295399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:03:30.243966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217554+0000","last_change":"2024-05-27T08:35:04.022731+0000","last_active":"2024-05-27T08:38:16.217554+0000","last_peered":"2024-05-27T08:38:16.217554+0000","last_clean":"2024-05-27T08:38:16.217554+0000","last_became_active":"2024-05-27T08:35:04.022349+0000","last_became_peered":"2024-05-27T08:35:04.022349+0000","last_unstale":"2024-05-27T08:38:16.217554+0000","last_undegraded":"2024-05-27T08:38:16.217554+0000","last_fullsized":"2024-05-27T08:38:16.217554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:38.651520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438589+0000","last_change":"2024-05-27T08:35:07.052623+0000","last_active":"2024-05-27T08:38:16.438589+0000","last_peered":"2024-05-27T08:38:16.438589+0000","last_clean":"2024-05-27T08:38:16.438589+0000","last_became_active":"2024-05-27T08:35:07.052313+0000","last_became_peered":"2024-05-27T08:35:07.052313+0000","last_unstale":"2024-05-27T08:38:16.438589+0000","last_undegraded":"2024-05-27T08:38:16.438589+0000","last_fullsized":"2024-05-27T08:38:16.438589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:24.612523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":214,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.536811+0000","last_change":"2024-05-27T08:35:04.023756+0000","last_active":"2024-05-27T08:38:16.536811+0000","last_peered":"2024-05-27T08:38:16.536811+0000","last_clean":"2024-05-27T08:38:16.536811+0000","last_became_active":"2024-05-27T08:35:04.023401+0000","last_became_peered":"2024-05-27T08:35:04.023401+0000","last_unstale":"2024-05-27T08:38:16.536811+0000","last_undegraded":"2024-05-27T08:38:16.536811+0000","last_fullsized":"2024-05-27T08:38:16.536811+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:34.887246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":179,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438391+0000","last_change":"2024-05-27T08:35:07.050720+0000","last_active":"2024-05-27T08:38:16.438391+0000","last_peered":"2024-05-27T08:38:16.438391+0000","last_clean":"2024-05-27T08:38:16.438391+0000","last_became_active":"2024-05-27T08:35:07.050135+0000","last_became_peered":"2024-05-27T08:35:07.050135+0000","last_unstale":"2024-05-27T08:38:16.438391+0000","last_undegraded":"2024-05-27T08:38:16.438391+0000","last_fullsized":"2024-05-27T08:38:16.438391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:40.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320069+0000","last_change":"2024-05-27T08:35:58.421002+0000","last_active":"2024-05-27T08:38:16.320069+0000","last_peered":"2024-05-27T08:38:16.320069+0000","last_clean":"2024-05-27T08:38:16.320069+0000","last_became_active":"2024-05-27T08:35:58.408889+0000","last_became_peered":"2024-05-27T08:35:58.408889+0000","last_unstale":"2024-05-27T08:38:16.320069+0000","last_undegraded":"2024-05-27T08:38:16.320069+0000","last_fullsized":"2024-05-27T08:38:16.320069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:29.705436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217489+0000","last_change":"2024-05-27T08:35:07.050504+0000","last_active":"2024-05-27T08:38:16.217489+0000","last_peered":"2024-05-27T08:38:16.217489+0000","last_clean":"2024-05-27T08:38:16.217489+0000","last_became_active":"2024-05-27T08:35:07.049368+0000","last_became_peered":"2024-05-27T08:35:07.049368+0000","last_unstale":"2024-05-27T08:38:16.217489+0000","last_undegraded":"2024-05-27T08:38:16.217489+0000","last_fullsized":"2024-05-27T08:38:16.217489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:52.996313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'2","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320461+0000","last_change":"2024-05-27T08:35:04.023924+0000","last_active":"2024-05-27T08:38:16.320461+0000","last_peered":"2024-05-27T08:38:16.320461+0000","last_clean":"2024-05-27T08:38:16.320461+0000","last_became_active":"2024-05-27T08:35:04.023548+0000","last_became_peered":"2024-05-27T08:35:04.023548+0000","last_unstale":"2024-05-27T08:38:16.320461+0000","last_undegraded":"2024-05-27T08:38:16.320461+0000","last_fullsized":"2024-05-27T08:38:16.320461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:50.441490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439437+0000","last_change":"2024-05-27T08:35:58.376335+0000","last_active":"2024-05-27T08:38:16.439437+0000","last_peered":"2024-05-27T08:38:16.439437+0000","last_clean":"2024-05-27T08:38:16.439437+0000","last_became_active":"2024-05-27T08:35:58.375926+0000","last_became_peered":"2024-05-27T08:35:58.375926+0000","last_unstale":"2024-05-27T08:38:16.439437+0000","last_undegraded":"2024-05-27T08:38:16.439437+0000","last_fullsized":"2024-05-27T08:38:16.439437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:33.892172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317833+0000","last_change":"2024-05-27T08:35:58.374310+0000","last_active":"2024-05-27T08:38:16.317833+0000","last_peered":"2024-05-27T08:38:16.317833+0000","last_clean":"2024-05-27T08:38:16.317833+0000","last_became_active":"2024-05-27T08:35:58.373929+0000","last_became_peered":"2024-05-27T08:35:58.373929+0000","last_unstale":"2024-05-27T08:38:16.317833+0000","last_undegraded":"2024-05-27T08:38:16.317833+0000","last_fullsized":"2024-05-27T08:38:16.317833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:20.901125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325887+0000","last_change":"2024-05-27T08:35:07.039182+0000","last_active":"2024-05-27T08:38:16.325887+0000","last_peered":"2024-05-27T08:38:16.325887+0000","last_clean":"2024-05-27T08:38:16.325887+0000","last_became_active":"2024-05-27T08:35:07.038793+0000","last_became_peered":"2024-05-27T08:35:07.038793+0000","last_unstale":"2024-05-27T08:38:16.325887+0000","last_undegraded":"2024-05-27T08:38:16.325887+0000","last_fullsized":"2024-05-27T08:38:16.325887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:03.839681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'7","reported_seq":211,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.552162+0000","last_change":"2024-05-27T08:35:04.023194+0000","last_active":"2024-05-27T08:38:16.552162+0000","last_peered":"2024-05-27T08:38:16.552162+0000","last_clean":"2024-05-27T08:38:16.552162+0000","last_became_active":"2024-05-27T08:35:04.022823+0000","last_became_peered":"2024-05-27T08:35:04.022823+0000","last_unstale":"2024-05-27T08:38:16.552162+0000","last_undegraded":"2024-05-27T08:38:16.552162+0000","last_fullsized":"2024-05-27T08:38:16.552162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:44:28.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325773+0000","last_change":"2024-05-27T08:35:07.050717+0000","last_active":"2024-05-27T08:38:16.325773+0000","last_peered":"2024-05-27T08:38:16.325773+0000","last_clean":"2024-05-27T08:38:16.325773+0000","last_became_active":"2024-05-27T08:35:07.050413+0000","last_became_peered":"2024-05-27T08:35:07.050413+0000","last_unstale":"2024-05-27T08:38:16.325773+0000","last_undegraded":"2024-05-27T08:38:16.325773+0000","last_fullsized":"2024-05-27T08:38:16.325773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:00:27.693141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217821+0000","last_change":"2024-05-27T08:35:07.053614+0000","last_active":"2024-05-27T08:38:16.217821+0000","last_peered":"2024-05-27T08:38:16.217821+0000","last_clean":"2024-05-27T08:38:16.217821+0000","last_became_active":"2024-05-27T08:35:07.053253+0000","last_became_peered":"2024-05-27T08:35:07.053253+0000","last_unstale":"2024-05-27T08:38:16.217821+0000","last_undegraded":"2024-05-27T08:38:16.217821+0000","last_fullsized":"2024-05-27T08:38:16.217821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:15:08.985984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":210,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.463537+0000","last_change":"2024-05-27T08:35:04.246500+0000","last_active":"2024-05-27T08:38:16.463537+0000","last_peered":"2024-05-27T08:38:16.463537+0000","last_clean":"2024-05-27T08:38:16.463537+0000","last_became_active":"2024-05-27T08:35:04.244575+0000","last_became_peered":"2024-05-27T08:35:04.244575+0000","last_unstale":"2024-05-27T08:38:16.463537+0000","last_undegraded":"2024-05-27T08:38:16.463537+0000","last_fullsized":"2024-05-27T08:38:16.463537+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:24:10.818057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218987+0000","last_change":"2024-05-27T08:35:07.048488+0000","last_active":"2024-05-27T08:38:16.218987+0000","last_peered":"2024-05-27T08:38:16.218987+0000","last_clean":"2024-05-27T08:38:16.218987+0000","last_became_active":"2024-05-27T08:35:07.048172+0000","last_became_peered":"2024-05-27T08:35:07.048172+0000","last_unstale":"2024-05-27T08:38:16.218987+0000","last_undegraded":"2024-05-27T08:38:16.218987+0000","last_fullsized":"2024-05-27T08:38:16.218987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:01.632944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255015+0000","last_change":"2024-05-27T08:35:04.022807+0000","last_active":"2024-05-27T08:38:16.255015+0000","last_peered":"2024-05-27T08:38:16.255015+0000","last_clean":"2024-05-27T08:38:16.255015+0000","last_became_active":"2024-05-27T08:35:04.021198+0000","last_became_peered":"2024-05-27T08:35:04.021198+0000","last_unstale":"2024-05-27T08:38:16.255015+0000","last_undegraded":"2024-05-27T08:38:16.255015+0000","last_fullsized":"2024-05-27T08:38:16.255015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:42:10.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":103,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438484+0000","last_change":"2024-05-27T08:33:45.683600+0000","last_active":"2024-05-27T08:38:16.438484+0000","last_peered":"2024-05-27T08:38:16.438484+0000","last_clean":"2024-05-27T08:38:16.438484+0000","last_became_active":"2024-05-27T08:33:45.677989+0000","last_became_peered":"2024-05-27T08:33:45.677989+0000","last_unstale":"2024-05-27T08:38:16.438484+0000","last_undegraded":"2024-05-27T08:38:16.438484+0000","last_fullsized":"2024-05-27T08:38:16.438484+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_clean_scrub_stamp":"2024-05-27T08:30:25.507727+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:48:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295270+0000","last_change":"2024-05-27T08:35:07.045879+0000","last_active":"2024-05-27T08:38:16.295270+0000","last_peered":"2024-05-27T08:38:16.295270+0000","last_clean":"2024-05-27T08:38:16.295270+0000","last_became_active":"2024-05-27T08:35:07.045522+0000","last_became_peered":"2024-05-27T08:35:07.045522+0000","last_unstale":"2024-05-27T08:38:16.295270+0000","last_undegraded":"2024-05-27T08:38:16.295270+0000","last_fullsized":"2024-05-27T08:38:16.295270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:30.393769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438619+0000","last_change":"2024-05-27T08:35:04.022419+0000","last_active":"2024-05-27T08:38:16.438619+0000","last_peered":"2024-05-27T08:38:16.438619+0000","last_clean":"2024-05-27T08:38:16.438619+0000","last_became_active":"2024-05-27T08:35:04.022039+0000","last_became_peered":"2024-05-27T08:35:04.022039+0000","last_unstale":"2024-05-27T08:38:16.438619+0000","last_undegraded":"2024-05-27T08:38:16.438619+0000","last_fullsized":"2024-05-27T08:38:16.438619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:31.260596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320374+0000","last_change":"2024-05-27T08:35:04.245214+0000","last_active":"2024-05-27T08:38:16.320374+0000","last_peered":"2024-05-27T08:38:16.320374+0000","last_clean":"2024-05-27T08:38:16.320374+0000","last_became_active":"2024-05-27T08:35:04.244768+0000","last_became_peered":"2024-05-27T08:35:04.244768+0000","last_unstale":"2024-05-27T08:38:16.320374+0000","last_undegraded":"2024-05-27T08:38:16.320374+0000","last_fullsized":"2024-05-27T08:38:16.320374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:34.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295585+0000","last_change":"2024-05-27T08:35:07.042721+0000","last_active":"2024-05-27T08:38:16.295585+0000","last_peered":"2024-05-27T08:38:16.295585+0000","last_clean":"2024-05-27T08:38:16.295585+0000","last_became_active":"2024-05-27T08:35:07.042435+0000","last_became_peered":"2024-05-27T08:35:07.042435+0000","last_unstale":"2024-05-27T08:38:16.295585+0000","last_undegraded":"2024-05-27T08:38:16.295585+0000","last_fullsized":"2024-05-27T08:38:16.295585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:49.186753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'52","reported_seq":103,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.424706+0000","last_change":"2024-05-27T08:35:04.017995+0000","last_active":"2024-05-27T08:38:16.424706+0000","last_peered":"2024-05-27T08:38:16.424706+0000","last_clean":"2024-05-27T08:38:16.424706+0000","last_became_active":"2024-05-27T08:35:04.017675+0000","last_became_peered":"2024-05-27T08:35:04.017675+0000","last_unstale":"2024-05-27T08:38:16.424706+0000","last_undegraded":"2024-05-27T08:38:16.424706+0000","last_fullsized":"2024-05-27T08:38:16.424706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:23:15.452624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219103+0000","last_change":"2024-05-27T08:35:07.046553+0000","last_active":"2024-05-27T08:38:16.219103+0000","last_peered":"2024-05-27T08:38:16.219103+0000","last_clean":"2024-05-27T08:38:16.219103+0000","last_became_active":"2024-05-27T08:35:07.046229+0000","last_became_peered":"2024-05-27T08:35:07.046229+0000","last_unstale":"2024-05-27T08:38:16.219103+0000","last_undegraded":"2024-05-27T08:38:16.219103+0000","last_fullsized":"2024-05-27T08:38:16.219103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:21.024307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255144+0000","last_change":"2024-05-27T08:35:04.262439+0000","last_active":"2024-05-27T08:38:16.255144+0000","last_peered":"2024-05-27T08:38:16.255144+0000","last_clean":"2024-05-27T08:38:16.255144+0000","last_became_active":"2024-05-27T08:35:04.261958+0000","last_became_peered":"2024-05-27T08:35:04.261958+0000","last_unstale":"2024-05-27T08:38:16.255144+0000","last_undegraded":"2024-05-27T08:38:16.255144+0000","last_fullsized":"2024-05-27T08:38:16.255144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:11:05.763068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439536+0000","last_change":"2024-05-27T08:35:07.042307+0000","last_active":"2024-05-27T08:38:16.439536+0000","last_peered":"2024-05-27T08:38:16.439536+0000","last_clean":"2024-05-27T08:38:16.439536+0000","last_became_active":"2024-05-27T08:35:07.042001+0000","last_became_peered":"2024-05-27T08:35:07.042001+0000","last_unstale":"2024-05-27T08:38:16.439536+0000","last_undegraded":"2024-05-27T08:38:16.439536+0000","last_fullsized":"2024-05-27T08:38:16.439536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:51.030157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219698+0000","last_change":"2024-05-27T08:35:04.245436+0000","last_active":"2024-05-27T08:38:16.219698+0000","last_peered":"2024-05-27T08:38:16.219698+0000","last_clean":"2024-05-27T08:38:16.219698+0000","last_became_active":"2024-05-27T08:35:04.245039+0000","last_became_peered":"2024-05-27T08:35:04.245039+0000","last_unstale":"2024-05-27T08:38:16.219698+0000","last_undegraded":"2024-05-27T08:38:16.219698+0000","last_fullsized":"2024-05-27T08:38:16.219698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:10:11.756864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320069+0000","last_change":"2024-05-27T08:35:07.051524+0000","last_active":"2024-05-27T08:38:16.320069+0000","last_peered":"2024-05-27T08:38:16.320069+0000","last_clean":"2024-05-27T08:38:16.320069+0000","last_became_active":"2024-05-27T08:35:07.050676+0000","last_became_peered":"2024-05-27T08:35:07.050676+0000","last_unstale":"2024-05-27T08:38:16.320069+0000","last_undegraded":"2024-05-27T08:38:16.320069+0000","last_fullsized":"2024-05-27T08:38:16.320069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:06.818977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218795+0000","last_change":"2024-05-27T08:35:04.022064+0000","last_active":"2024-05-27T08:38:16.218795+0000","last_peered":"2024-05-27T08:38:16.218795+0000","last_clean":"2024-05-27T08:38:16.218795+0000","last_became_active":"2024-05-27T08:35:04.021691+0000","last_became_peered":"2024-05-27T08:35:04.021691+0000","last_unstale":"2024-05-27T08:38:16.218795+0000","last_undegraded":"2024-05-27T08:38:16.218795+0000","last_fullsized":"2024-05-27T08:38:16.218795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:54.557756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439157+0000","last_change":"2024-05-27T08:35:07.049666+0000","last_active":"2024-05-27T08:38:16.439157+0000","last_peered":"2024-05-27T08:38:16.439157+0000","last_clean":"2024-05-27T08:38:16.439157+0000","last_became_active":"2024-05-27T08:35:07.049312+0000","last_became_peered":"2024-05-27T08:35:07.049312+0000","last_unstale":"2024-05-27T08:38:16.439157+0000","last_undegraded":"2024-05-27T08:38:16.439157+0000","last_fullsized":"2024-05-27T08:38:16.439157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:37:53.704582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'3","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438548+0000","last_change":"2024-05-27T08:35:04.018457+0000","last_active":"2024-05-27T08:38:16.438548+0000","last_peered":"2024-05-27T08:38:16.438548+0000","last_clean":"2024-05-27T08:38:16.438548+0000","last_became_active":"2024-05-27T08:35:04.018008+0000","last_became_peered":"2024-05-27T08:35:04.018008+0000","last_unstale":"2024-05-27T08:38:16.438548+0000","last_undegraded":"2024-05-27T08:38:16.438548+0000","last_fullsized":"2024-05-27T08:38:16.438548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:08:13.228725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219211+0000","last_change":"2024-05-27T08:35:07.047040+0000","last_active":"2024-05-27T08:38:16.219211+0000","last_peered":"2024-05-27T08:38:16.219211+0000","last_clean":"2024-05-27T08:38:16.219211+0000","last_became_active":"2024-05-27T08:35:07.046731+0000","last_became_peered":"2024-05-27T08:35:07.046731+0000","last_unstale":"2024-05-27T08:38:16.219211+0000","last_undegraded":"2024-05-27T08:38:16.219211+0000","last_fullsized":"2024-05-27T08:38:16.219211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:25.368349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320164+0000","last_change":"2024-05-27T08:35:04.022804+0000","last_active":"2024-05-27T08:38:16.320164+0000","last_peered":"2024-05-27T08:38:16.320164+0000","last_clean":"2024-05-27T08:38:16.320164+0000","last_became_active":"2024-05-27T08:35:04.022458+0000","last_became_peered":"2024-05-27T08:35:04.022458+0000","last_unstale":"2024-05-27T08:38:16.320164+0000","last_undegraded":"2024-05-27T08:38:16.320164+0000","last_fullsized":"2024-05-27T08:38:16.320164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:01.029226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438657+0000","last_change":"2024-05-27T08:35:07.049136+0000","last_active":"2024-05-27T08:38:16.438657+0000","last_peered":"2024-05-27T08:38:16.438657+0000","last_clean":"2024-05-27T08:38:16.438657+0000","last_became_active":"2024-05-27T08:35:07.048793+0000","last_became_peered":"2024-05-27T08:35:07.048793+0000","last_unstale":"2024-05-27T08:38:16.438657+0000","last_undegraded":"2024-05-27T08:38:16.438657+0000","last_fullsized":"2024-05-27T08:38:16.438657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:10:21.191743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":210,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.520790+0000","last_change":"2024-05-27T08:35:04.262102+0000","last_active":"2024-05-27T08:38:16.520790+0000","last_peered":"2024-05-27T08:38:16.520790+0000","last_clean":"2024-05-27T08:38:16.520790+0000","last_became_active":"2024-05-27T08:35:04.261665+0000","last_became_peered":"2024-05-27T08:35:04.261665+0000","last_unstale":"2024-05-27T08:38:16.520790+0000","last_undegraded":"2024-05-27T08:38:16.520790+0000","last_fullsized":"2024-05-27T08:38:16.520790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:57:57.712197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":176,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438962+0000","last_change":"2024-05-27T08:35:07.051557+0000","last_active":"2024-05-27T08:38:16.438962+0000","last_peered":"2024-05-27T08:38:16.438962+0000","last_clean":"2024-05-27T08:38:16.438962+0000","last_became_active":"2024-05-27T08:35:07.051246+0000","last_became_peered":"2024-05-27T08:35:07.051246+0000","last_unstale":"2024-05-27T08:38:16.438962+0000","last_undegraded":"2024-05-27T08:38:16.438962+0000","last_fullsized":"2024-05-27T08:38:16.438962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:08:13.248301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'41","reported_seq":77,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.422467+0000","last_change":"2024-05-27T08:35:04.022328+0000","last_active":"2024-05-27T08:38:16.422467+0000","last_peered":"2024-05-27T08:38:16.422467+0000","last_clean":"2024-05-27T08:38:16.422467+0000","last_became_active":"2024-05-27T08:35:04.021711+0000","last_became_peered":"2024-05-27T08:35:04.021711+0000","last_unstale":"2024-05-27T08:38:16.422467+0000","last_undegraded":"2024-05-27T08:38:16.422467+0000","last_fullsized":"2024-05-27T08:38:16.422467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:32.841474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438669+0000","last_change":"2024-05-27T08:35:07.056768+0000","last_active":"2024-05-27T08:38:16.438669+0000","last_peered":"2024-05-27T08:38:16.438669+0000","last_clean":"2024-05-27T08:38:16.438669+0000","last_became_active":"2024-05-27T08:35:07.056482+0000","last_became_peered":"2024-05-27T08:35:07.056482+0000","last_unstale":"2024-05-27T08:38:16.438669+0000","last_undegraded":"2024-05-27T08:38:16.438669+0000","last_fullsized":"2024-05-27T08:38:16.438669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:23.515625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295576+0000","last_change":"2024-05-27T08:35:04.016613+0000","last_active":"2024-05-27T08:38:16.295576+0000","last_peered":"2024-05-27T08:38:16.295576+0000","last_clean":"2024-05-27T08:38:16.295576+0000","last_became_active":"2024-05-27T08:35:04.016268+0000","last_became_peered":"2024-05-27T08:35:04.016268+0000","last_unstale":"2024-05-27T08:38:16.295576+0000","last_undegraded":"2024-05-27T08:38:16.295576+0000","last_fullsized":"2024-05-27T08:38:16.295576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:11.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438981+0000","last_change":"2024-05-27T08:35:07.051515+0000","last_active":"2024-05-27T08:38:16.438981+0000","last_peered":"2024-05-27T08:38:16.438981+0000","last_clean":"2024-05-27T08:38:16.438981+0000","last_became_active":"2024-05-27T08:35:07.051197+0000","last_became_peered":"2024-05-27T08:35:07.051197+0000","last_unstale":"2024-05-27T08:38:16.438981+0000","last_undegraded":"2024-05-27T08:38:16.438981+0000","last_fullsized":"2024-05-27T08:38:16.438981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:49.233474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439171+0000","last_change":"2024-05-27T08:35:07.051960+0000","last_active":"2024-05-27T08:38:16.439171+0000","last_peered":"2024-05-27T08:38:16.439171+0000","last_clean":"2024-05-27T08:38:16.439171+0000","last_became_active":"2024-05-27T08:35:07.051395+0000","last_became_peered":"2024-05-27T08:35:07.051395+0000","last_unstale":"2024-05-27T08:38:16.439171+0000","last_undegraded":"2024-05-27T08:38:16.439171+0000","last_fullsized":"2024-05-27T08:38:16.439171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:57.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438517+0000","last_change":"2024-05-27T08:35:07.050096+0000","last_active":"2024-05-27T08:38:16.438517+0000","last_peered":"2024-05-27T08:38:16.438517+0000","last_clean":"2024-05-27T08:38:16.438517+0000","last_became_active":"2024-05-27T08:35:07.049755+0000","last_became_peered":"2024-05-27T08:35:07.049755+0000","last_unstale":"2024-05-27T08:38:16.438517+0000","last_undegraded":"2024-05-27T08:38:16.438517+0000","last_fullsized":"2024-05-27T08:38:16.438517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:39.768128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438481+0000","last_change":"2024-05-27T08:35:07.048281+0000","last_active":"2024-05-27T08:38:16.438481+0000","last_peered":"2024-05-27T08:38:16.438481+0000","last_clean":"2024-05-27T08:38:16.438481+0000","last_became_active":"2024-05-27T08:35:07.047879+0000","last_became_peered":"2024-05-27T08:35:07.047879+0000","last_unstale":"2024-05-27T08:38:16.438481+0000","last_undegraded":"2024-05-27T08:38:16.438481+0000","last_fullsized":"2024-05-27T08:38:16.438481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:43.631671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438821+0000","last_change":"2024-05-27T08:35:07.050501+0000","last_active":"2024-05-27T08:38:16.438821+0000","last_peered":"2024-05-27T08:38:16.438821+0000","last_clean":"2024-05-27T08:38:16.438821+0000","last_became_active":"2024-05-27T08:35:07.050159+0000","last_became_peered":"2024-05-27T08:35:07.050159+0000","last_unstale":"2024-05-27T08:38:16.438821+0000","last_undegraded":"2024-05-27T08:38:16.438821+0000","last_fullsized":"2024-05-27T08:38:16.438821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:17:23.293019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295514+0000","last_change":"2024-05-27T08:35:58.372446+0000","last_active":"2024-05-27T08:38:16.295514+0000","last_peered":"2024-05-27T08:38:16.295514+0000","last_clean":"2024-05-27T08:38:16.295514+0000","last_became_active":"2024-05-27T08:35:58.372080+0000","last_became_peered":"2024-05-27T08:35:58.372080+0000","last_unstale":"2024-05-27T08:38:16.295514+0000","last_undegraded":"2024-05-27T08:38:16.295514+0000","last_fullsized":"2024-05-27T08:38:16.295514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:57.828720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439444+0000","last_change":"2024-05-27T08:35:07.052382+0000","last_active":"2024-05-27T08:38:16.439444+0000","last_peered":"2024-05-27T08:38:16.439444+0000","last_clean":"2024-05-27T08:38:16.439444+0000","last_became_active":"2024-05-27T08:35:07.052049+0000","last_became_peered":"2024-05-27T08:35:07.052049+0000","last_unstale":"2024-05-27T08:38:16.439444+0000","last_undegraded":"2024-05-27T08:38:16.439444+0000","last_fullsized":"2024-05-27T08:38:16.439444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:08.912872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438897+0000","last_change":"2024-05-27T08:35:07.043810+0000","last_active":"2024-05-27T08:38:16.438897+0000","last_peered":"2024-05-27T08:38:16.438897+0000","last_clean":"2024-05-27T08:38:16.438897+0000","last_became_active":"2024-05-27T08:35:07.043449+0000","last_became_peered":"2024-05-27T08:35:07.043449+0000","last_unstale":"2024-05-27T08:38:16.438897+0000","last_undegraded":"2024-05-27T08:38:16.438897+0000","last_fullsized":"2024-05-27T08:38:16.438897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:18.089405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218995+0000","last_change":"2024-05-27T08:35:58.371407+0000","last_active":"2024-05-27T08:38:16.218995+0000","last_peered":"2024-05-27T08:38:16.218995+0000","last_clean":"2024-05-27T08:38:16.218995+0000","last_became_active":"2024-05-27T08:35:58.370963+0000","last_became_peered":"2024-05-27T08:35:58.370963+0000","last_unstale":"2024-05-27T08:38:16.218995+0000","last_undegraded":"2024-05-27T08:38:16.218995+0000","last_fullsized":"2024-05-27T08:38:16.218995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:23.458573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217613+0000","last_change":"2024-05-27T08:35:07.049960+0000","last_active":"2024-05-27T08:38:16.217613+0000","last_peered":"2024-05-27T08:38:16.217613+0000","last_clean":"2024-05-27T08:38:16.217613+0000","last_became_active":"2024-05-27T08:35:07.049595+0000","last_became_peered":"2024-05-27T08:35:07.049595+0000","last_unstale":"2024-05-27T08:38:16.217613+0000","last_undegraded":"2024-05-27T08:38:16.217613+0000","last_fullsized":"2024-05-27T08:38:16.217613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:18.182658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218743+0000","last_change":"2024-05-27T08:35:07.043790+0000","last_active":"2024-05-27T08:38:16.218743+0000","last_peered":"2024-05-27T08:38:16.218743+0000","last_clean":"2024-05-27T08:38:16.218743+0000","last_became_active":"2024-05-27T08:35:07.043476+0000","last_became_peered":"2024-05-27T08:35:07.043476+0000","last_unstale":"2024-05-27T08:38:16.218743+0000","last_undegraded":"2024-05-27T08:38:16.218743+0000","last_fullsized":"2024-05-27T08:38:16.218743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:01.626979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219848+0000","last_change":"2024-05-27T08:35:07.044928+0000","last_active":"2024-05-27T08:38:16.219848+0000","last_peered":"2024-05-27T08:38:16.219848+0000","last_clean":"2024-05-27T08:38:16.219848+0000","last_became_active":"2024-05-27T08:35:07.044577+0000","last_became_peered":"2024-05-27T08:35:07.044577+0000","last_unstale":"2024-05-27T08:38:16.219848+0000","last_undegraded":"2024-05-27T08:38:16.219848+0000","last_fullsized":"2024-05-27T08:38:16.219848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:36.560525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295715+0000","last_change":"2024-05-27T08:35:07.045067+0000","last_active":"2024-05-27T08:38:16.295715+0000","last_peered":"2024-05-27T08:38:16.295715+0000","last_clean":"2024-05-27T08:38:16.295715+0000","last_became_active":"2024-05-27T08:35:07.044759+0000","last_became_peered":"2024-05-27T08:35:07.044759+0000","last_unstale":"2024-05-27T08:38:16.295715+0000","last_undegraded":"2024-05-27T08:38:16.295715+0000","last_fullsized":"2024-05-27T08:38:16.295715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:08.440135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439347+0000","last_change":"2024-05-27T08:35:07.048186+0000","last_active":"2024-05-27T08:38:16.439347+0000","last_peered":"2024-05-27T08:38:16.439347+0000","last_clean":"2024-05-27T08:38:16.439347+0000","last_became_active":"2024-05-27T08:35:07.047828+0000","last_became_peered":"2024-05-27T08:35:07.047828+0000","last_unstale":"2024-05-27T08:38:16.439347+0000","last_undegraded":"2024-05-27T08:38:16.439347+0000","last_fullsized":"2024-05-27T08:38:16.439347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:09.928234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600648,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1867,"num_read_kb":1047,"num_write":379,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2371584,"data_stored":1998576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":306,"ondisk_log_size":306,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":458064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743482,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1654784,"data_stored":1114344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939711,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32580,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712572,"statfs":{"total":95995035648,"available":95961673728,"internally_reserved":0,"allocated":1368064,"data_stored":850237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135940,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28340,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716812,"statfs":{"total":95995035648,"available":95966015488,"internally_reserved":0,"allocated":1351680,"data_stored":849361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332168,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1552384,"data_stored":1041942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495694,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":958464,"data_stored":457437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":525496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":458468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":525547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117085,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37116,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708036,"statfs":{"total":95995035648,"available":95957028864,"internally_reserved":0,"allocated":1699840,"data_stored":1178011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920855,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":457634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691924,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1716224,"data_stored":1181727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":200733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T08:38:24.312 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-27T08:38:24.312 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-05-27T08:38:24.552 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:25.010 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:24 smithi077 ceph-mon[32007]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:24 smithi179 ceph-mon[36481]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:25.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:24 smithi092 ceph-mon[36049]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:25.970 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:25 smithi077 ceph-mon[32007]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-05-27T08:38:26.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:25 smithi179 ceph-mon[36481]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-05-27T08:38:26.185 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 -- 172.21.15.77:0/3529218976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec41095b0 msgr2=0x7fbec4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 --2- 172.21.15.77:0/3529218976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec41095b0 0x7fbec4109990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009990 tx=0x7fbebc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 -- 172.21.15.77:0/3529218976 shutdown_connections 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 --2- 172.21.15.77:0/3529218976 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec406bbe0 0x7fbec41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 --2- 172.21.15.77:0/3529218976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec41095b0 0x7fbec4109990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 --2- 172.21.15.77:0/3529218976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbec406b220 0x7fbec406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 -- 172.21.15.77:0/3529218976 >> 172.21.15.77:0/3529218976 conn(0x7fbec40793f0 msgr2=0x7fbec4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 -- 172.21.15.77:0/3529218976 shutdown_connections 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.183+0000 7fbecc690700 1 -- 172.21.15.77:0/3529218976 wait complete. 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecc690700 1 Processor -- start 2024-05-27T08:38:26.186 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecc690700 1 -- start start 2024-05-27T08:38:26.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecc690700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec406b220 0x7fbec4199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:26.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecc690700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbec406bbe0 0x7fbec419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:26.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecc690700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec41095b0 0x7fbec419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:26.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecc690700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbec41137c0 con 0x7fbec406b220 2024-05-27T08:38:26.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecc690700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fbec4113640 con 0x7fbec41095b0 2024-05-27T08:38:26.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecc690700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbec4113940 con 0x7fbec406bbe0 2024-05-27T08:38:26.188 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbeca42c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec406b220 0x7fbec4199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:26.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbeca42c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec406b220 0x7fbec4199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39806/0 (socket says 172.21.15.77:39806) 2024-05-27T08:38:26.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbeca42c700 1 -- 172.21.15.77:0/2460257547 learned_addr learned my addr 172.21.15.77:0/2460257547 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:26.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecac2d700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec41095b0 0x7fbec419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:26.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbec9c2b700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbec406bbe0 0x7fbec419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:26.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbeca42c700 1 -- 172.21.15.77:0/2460257547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbec406bbe0 msgr2=0x7fbec419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:26.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbeca42c700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbec406bbe0 0x7fbec419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.189 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbeca42c700 1 -- 172.21.15.77:0/2460257547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec41095b0 msgr2=0x7fbec419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbeca42c700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec41095b0 0x7fbec419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbeca42c700 1 -- 172.21.15.77:0/2460257547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec419ed40 con 0x7fbec406b220 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.184+0000 7fbecac2d700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec41095b0 0x7fbec419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.185+0000 7fbeca42c700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec406b220 0x7fbec4199d30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbebc000c00 tx=0x7fbebc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.185+0000 7fbec37fe700 1 -- 172.21.15.77:0/2460257547 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc027070 con 0x7fbec406b220 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.185+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec419efd0 con 0x7fbec406b220 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.185+0000 7fbec37fe700 1 -- 172.21.15.77:0/2460257547 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbebc0041d0 con 0x7fbec406b220 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.185+0000 7fbec37fe700 1 -- 172.21.15.77:0/2460257547 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc00f460 con 0x7fbec406b220 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.185+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec41a68b0 con 0x7fbec406b220 2024-05-27T08:38:26.190 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.186+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb0005330 con 0x7fbec406b220 2024-05-27T08:38:26.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.187+0000 7fbec37fe700 1 -- 172.21.15.77:0/2460257547 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fbebc02a070 con 0x7fbec406b220 2024-05-27T08:38:26.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.188+0000 7fbec37fe700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbeac0b2a80 0x7fbeac0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:26.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.188+0000 7fbec9c2b700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbeac0b2a80 0x7fbeac0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:26.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.188+0000 7fbec37fe700 1 -- 172.21.15.77:0/2460257547 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fbebc0e3aa0 con 0x7fbec406b220 2024-05-27T08:38:26.191 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.188+0000 7fbec9c2b700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbeac0b2a80 0x7fbeac0b4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8006fd0 tx=0x7fbeb8008040 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:26.193 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.191+0000 7fbec37fe700 1 -- 172.21.15.77:0/2460257547 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbebc031070 con 0x7fbec406b220 2024-05-27T08:38:26.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:25 smithi092 ceph-mon[36049]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 72 op/s 2024-05-27T08:38:26.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.401+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbeb0000bc0 con 0x7fbeac0b2a80 2024-05-27T08:38:26.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.408+0000 7fbec37fe700 1 -- 172.21.15.77:0/2460257547 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257814 (secure 0 0 0) 0x7fbeb0000bc0 con 0x7fbeac0b2a80 2024-05-27T08:38:26.413 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbeac0b2a80 msgr2=0x7fbeac0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbeac0b2a80 0x7fbeac0b4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8006fd0 tx=0x7fbeb8008040 comp rx=0 tx=0).stop 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec406b220 msgr2=0x7fbec4199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec406b220 0x7fbec4199d30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbebc000c00 tx=0x7fbebc0047b0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 shutdown_connections 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fbeac0b2a80 0x7fbeac0b4f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbec41095b0 0x7fbec419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbec406b220 0x7fbec4199d30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 --2- 172.21.15.77:0/2460257547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbec406bbe0 0x7fbec419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 >> 172.21.15.77:0/2460257547 conn(0x7fbec40793f0 msgr2=0x7fbec41012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:26.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 shutdown_connections 2024-05-27T08:38:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:26.411+0000 7fbecc690700 1 -- 172.21.15.77:0/2460257547 wait complete. 2024-05-27T08:38:26.416 INFO:teuthology.orchestra.run.smithi077.stderr:dumped all 2024-05-27T08:38:27.082 INFO:teuthology.orchestra.run.smithi077.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-05-27T08:38:26.233665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387882,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2314,"num_read_kb":1550,"num_write":646,"num_write_kb":4536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2953206,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":988,"ondisk_log_size":988,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":377476,"kb_used_data":15456,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124564348,"statfs":{"total":1151940427776,"available":1151553892352,"internally_reserved":0,"allocated":15826944,"data_stored":9634434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":366,"num_read_kb":434,"num_write":142,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991698"},"pg_stats":[{"pgid":"3.3f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439549+0000","last_change":"2024-05-27T08:35:07.046361+0000","last_active":"2024-05-27T08:38:16.439549+0000","last_peered":"2024-05-27T08:38:16.439549+0000","last_clean":"2024-05-27T08:38:16.439549+0000","last_became_active":"2024-05-27T08:35:07.045954+0000","last_became_peered":"2024-05-27T08:35:07.045954+0000","last_unstale":"2024-05-27T08:38:16.439549+0000","last_undegraded":"2024-05-27T08:38:16.439549+0000","last_fullsized":"2024-05-27T08:38:16.439549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:03:47.458964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"85'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439255+0000","last_change":"2024-05-27T08:35:07.043725+0000","last_active":"2024-05-27T08:38:16.439255+0000","last_peered":"2024-05-27T08:38:16.439255+0000","last_clean":"2024-05-27T08:38:16.439255+0000","last_became_active":"2024-05-27T08:35:07.043447+0000","last_became_peered":"2024-05-27T08:35:07.043447+0000","last_unstale":"2024-05-27T08:38:16.439255+0000","last_undegraded":"2024-05-27T08:38:16.439255+0000","last_fullsized":"2024-05-27T08:38:16.439255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:00:44.509202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439075+0000","last_change":"2024-05-27T08:35:07.048075+0000","last_active":"2024-05-27T08:38:16.439075+0000","last_peered":"2024-05-27T08:38:16.439075+0000","last_clean":"2024-05-27T08:38:16.439075+0000","last_became_active":"2024-05-27T08:35:07.047687+0000","last_became_peered":"2024-05-27T08:35:07.047687+0000","last_unstale":"2024-05-27T08:38:16.439075+0000","last_undegraded":"2024-05-27T08:38:16.439075+0000","last_fullsized":"2024-05-27T08:38:16.439075+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:05:15.197231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255117+0000","last_change":"2024-05-27T08:35:07.046809+0000","last_active":"2024-05-27T08:38:16.255117+0000","last_peered":"2024-05-27T08:38:16.255117+0000","last_clean":"2024-05-27T08:38:16.255117+0000","last_became_active":"2024-05-27T08:35:07.046453+0000","last_became_peered":"2024-05-27T08:35:07.046453+0000","last_unstale":"2024-05-27T08:38:16.255117+0000","last_undegraded":"2024-05-27T08:38:16.255117+0000","last_fullsized":"2024-05-27T08:38:16.255117+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:41:43.209613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"85'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320175+0000","last_change":"2024-05-27T08:35:07.040242+0000","last_active":"2024-05-27T08:38:16.320175+0000","last_peered":"2024-05-27T08:38:16.320175+0000","last_clean":"2024-05-27T08:38:16.320175+0000","last_became_active":"2024-05-27T08:35:07.039860+0000","last_became_peered":"2024-05-27T08:35:07.039860+0000","last_unstale":"2024-05-27T08:38:16.320175+0000","last_undegraded":"2024-05-27T08:38:16.320175+0000","last_fullsized":"2024-05-27T08:38:16.320175+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:34:49.212799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325893+0000","last_change":"2024-05-27T08:35:07.048176+0000","last_active":"2024-05-27T08:38:16.325893+0000","last_peered":"2024-05-27T08:38:16.325893+0000","last_clean":"2024-05-27T08:38:16.325893+0000","last_became_active":"2024-05-27T08:35:07.047860+0000","last_became_peered":"2024-05-27T08:35:07.047860+0000","last_unstale":"2024-05-27T08:38:16.325893+0000","last_undegraded":"2024-05-27T08:38:16.325893+0000","last_fullsized":"2024-05-27T08:38:16.325893+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:56:19.459704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438445+0000","last_change":"2024-05-27T08:35:07.052285+0000","last_active":"2024-05-27T08:38:16.438445+0000","last_peered":"2024-05-27T08:38:16.438445+0000","last_clean":"2024-05-27T08:38:16.438445+0000","last_became_active":"2024-05-27T08:35:07.051970+0000","last_became_peered":"2024-05-27T08:35:07.051970+0000","last_unstale":"2024-05-27T08:38:16.438445+0000","last_undegraded":"2024-05-27T08:38:16.438445+0000","last_fullsized":"2024-05-27T08:38:16.438445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:56.543022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218781+0000","last_change":"2024-05-27T08:35:07.047527+0000","last_active":"2024-05-27T08:38:16.218781+0000","last_peered":"2024-05-27T08:38:16.218781+0000","last_clean":"2024-05-27T08:38:16.218781+0000","last_became_active":"2024-05-27T08:35:07.047204+0000","last_became_peered":"2024-05-27T08:35:07.047204+0000","last_unstale":"2024-05-27T08:38:16.218781+0000","last_undegraded":"2024-05-27T08:38:16.218781+0000","last_fullsized":"2024-05-27T08:38:16.218781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:04:43.547847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.220295+0000","last_change":"2024-05-27T08:35:07.049108+0000","last_active":"2024-05-27T08:38:16.220295+0000","last_peered":"2024-05-27T08:38:16.220295+0000","last_clean":"2024-05-27T08:38:16.220295+0000","last_became_active":"2024-05-27T08:35:07.048540+0000","last_became_peered":"2024-05-27T08:35:07.048540+0000","last_unstale":"2024-05-27T08:38:16.220295+0000","last_undegraded":"2024-05-27T08:38:16.220295+0000","last_fullsized":"2024-05-27T08:38:16.220295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:20:56.219362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"85'16","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.326127+0000","last_change":"2024-05-27T08:35:07.049775+0000","last_active":"2024-05-27T08:38:16.326127+0000","last_peered":"2024-05-27T08:38:16.326127+0000","last_clean":"2024-05-27T08:38:16.326127+0000","last_became_active":"2024-05-27T08:35:07.049466+0000","last_became_peered":"2024-05-27T08:35:07.049466+0000","last_unstale":"2024-05-27T08:38:16.326127+0000","last_undegraded":"2024-05-27T08:38:16.326127+0000","last_fullsized":"2024-05-27T08:38:16.326127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:18:30.394390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438881+0000","last_change":"2024-05-27T08:35:07.050870+0000","last_active":"2024-05-27T08:38:16.438881+0000","last_peered":"2024-05-27T08:38:16.438881+0000","last_clean":"2024-05-27T08:38:16.438881+0000","last_became_active":"2024-05-27T08:35:07.050527+0000","last_became_peered":"2024-05-27T08:35:07.050527+0000","last_unstale":"2024-05-27T08:38:16.438881+0000","last_undegraded":"2024-05-27T08:38:16.438881+0000","last_fullsized":"2024-05-27T08:38:16.438881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:56:41.477053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.254986+0000","last_change":"2024-05-27T08:35:07.052955+0000","last_active":"2024-05-27T08:38:16.254986+0000","last_peered":"2024-05-27T08:38:16.254986+0000","last_clean":"2024-05-27T08:38:16.254986+0000","last_became_active":"2024-05-27T08:35:07.052500+0000","last_became_peered":"2024-05-27T08:35:07.052500+0000","last_unstale":"2024-05-27T08:38:16.254986+0000","last_undegraded":"2024-05-27T08:38:16.254986+0000","last_fullsized":"2024-05-27T08:38:16.254986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:01:25.093408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"85'10","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295540+0000","last_change":"2024-05-27T08:35:07.044866+0000","last_active":"2024-05-27T08:38:16.295540+0000","last_peered":"2024-05-27T08:38:16.295540+0000","last_clean":"2024-05-27T08:38:16.295540+0000","last_became_active":"2024-05-27T08:35:07.044554+0000","last_became_peered":"2024-05-27T08:35:07.044554+0000","last_unstale":"2024-05-27T08:38:16.295540+0000","last_undegraded":"2024-05-27T08:38:16.295540+0000","last_fullsized":"2024-05-27T08:38:16.295540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:14:45.624970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438667+0000","last_change":"2024-05-27T08:35:07.043985+0000","last_active":"2024-05-27T08:38:16.438667+0000","last_peered":"2024-05-27T08:38:16.438667+0000","last_clean":"2024-05-27T08:38:16.438667+0000","last_became_active":"2024-05-27T08:35:07.043694+0000","last_became_peered":"2024-05-27T08:35:07.043694+0000","last_unstale":"2024-05-27T08:38:16.438667+0000","last_undegraded":"2024-05-27T08:38:16.438667+0000","last_fullsized":"2024-05-27T08:38:16.438667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:22:49.796670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438460+0000","last_change":"2024-05-27T08:35:07.045542+0000","last_active":"2024-05-27T08:38:16.438460+0000","last_peered":"2024-05-27T08:38:16.438460+0000","last_clean":"2024-05-27T08:38:16.438460+0000","last_became_active":"2024-05-27T08:35:07.045209+0000","last_became_peered":"2024-05-27T08:35:07.045209+0000","last_unstale":"2024-05-27T08:38:16.438460+0000","last_undegraded":"2024-05-27T08:38:16.438460+0000","last_fullsized":"2024-05-27T08:38:16.438460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:18:57.733089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219687+0000","last_change":"2024-05-27T08:35:07.047193+0000","last_active":"2024-05-27T08:38:16.219687+0000","last_peered":"2024-05-27T08:38:16.219687+0000","last_clean":"2024-05-27T08:38:16.219687+0000","last_became_active":"2024-05-27T08:35:07.046862+0000","last_became_peered":"2024-05-27T08:35:07.046862+0000","last_unstale":"2024-05-27T08:38:16.219687+0000","last_undegraded":"2024-05-27T08:38:16.219687+0000","last_fullsized":"2024-05-27T08:38:16.219687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:32:21.288642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439576+0000","last_change":"2024-05-27T08:35:07.045452+0000","last_active":"2024-05-27T08:38:16.439576+0000","last_peered":"2024-05-27T08:38:16.439576+0000","last_clean":"2024-05-27T08:38:16.439576+0000","last_became_active":"2024-05-27T08:35:07.044783+0000","last_became_peered":"2024-05-27T08:35:07.044783+0000","last_unstale":"2024-05-27T08:38:16.439576+0000","last_undegraded":"2024-05-27T08:38:16.439576+0000","last_fullsized":"2024-05-27T08:38:16.439576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:28:03.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317955+0000","last_change":"2024-05-27T08:35:07.042864+0000","last_active":"2024-05-27T08:38:16.317955+0000","last_peered":"2024-05-27T08:38:16.317955+0000","last_clean":"2024-05-27T08:38:16.317955+0000","last_became_active":"2024-05-27T08:35:07.042523+0000","last_became_peered":"2024-05-27T08:35:07.042523+0000","last_unstale":"2024-05-27T08:38:16.317955+0000","last_undegraded":"2024-05-27T08:38:16.317955+0000","last_fullsized":"2024-05-27T08:38:16.317955+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:34.275977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438978+0000","last_change":"2024-05-27T08:35:07.042667+0000","last_active":"2024-05-27T08:38:16.438978+0000","last_peered":"2024-05-27T08:38:16.438978+0000","last_clean":"2024-05-27T08:38:16.438978+0000","last_became_active":"2024-05-27T08:35:07.042090+0000","last_became_peered":"2024-05-27T08:35:07.042090+0000","last_unstale":"2024-05-27T08:38:16.438978+0000","last_undegraded":"2024-05-27T08:38:16.438978+0000","last_fullsized":"2024-05-27T08:38:16.438978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:45:28.764641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'14","reported_seq":241,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:20.556750+0000","last_change":"2024-05-27T08:35:58.391406+0000","last_active":"2024-05-27T08:38:20.556750+0000","last_peered":"2024-05-27T08:38:20.556750+0000","last_clean":"2024-05-27T08:38:20.556750+0000","last_became_active":"2024-05-27T08:35:58.376035+0000","last_became_peered":"2024-05-27T08:35:58.376035+0000","last_unstale":"2024-05-27T08:38:20.556750+0000","last_undegraded":"2024-05-27T08:38:20.556750+0000","last_fullsized":"2024-05-27T08:38:20.556750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:31:54.751467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"85'5","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317866+0000","last_change":"2024-05-27T08:35:58.376043+0000","last_active":"2024-05-27T08:38:16.317866+0000","last_peered":"2024-05-27T08:38:16.317866+0000","last_clean":"2024-05-27T08:38:16.317866+0000","last_became_active":"2024-05-27T08:35:58.375695+0000","last_became_peered":"2024-05-27T08:35:58.375695+0000","last_unstale":"2024-05-27T08:38:16.317866+0000","last_undegraded":"2024-05-27T08:38:16.317866+0000","last_fullsized":"2024-05-27T08:38:16.317866+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:30:40.875555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1e","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219253+0000","last_change":"2024-05-27T08:35:04.245013+0000","last_active":"2024-05-27T08:38:16.219253+0000","last_peered":"2024-05-27T08:38:16.219253+0000","last_clean":"2024-05-27T08:38:16.219253+0000","last_became_active":"2024-05-27T08:35:04.244629+0000","last_became_peered":"2024-05-27T08:35:04.244629+0000","last_unstale":"2024-05-27T08:38:16.219253+0000","last_undegraded":"2024-05-27T08:38:16.219253+0000","last_fullsized":"2024-05-27T08:38:16.219253+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:41:43.747592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439484+0000","last_change":"2024-05-27T08:35:07.041776+0000","last_active":"2024-05-27T08:38:16.439484+0000","last_peered":"2024-05-27T08:38:16.439484+0000","last_clean":"2024-05-27T08:38:16.439484+0000","last_became_active":"2024-05-27T08:35:07.041400+0000","last_became_peered":"2024-05-27T08:35:07.041400+0000","last_unstale":"2024-05-27T08:38:16.439484+0000","last_undegraded":"2024-05-27T08:38:16.439484+0000","last_fullsized":"2024-05-27T08:38:16.439484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:27:10.091310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"83'12","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320491+0000","last_change":"2024-05-27T08:35:04.023072+0000","last_active":"2024-05-27T08:38:16.320491+0000","last_peered":"2024-05-27T08:38:16.320491+0000","last_clean":"2024-05-27T08:38:16.320491+0000","last_became_active":"2024-05-27T08:35:04.022232+0000","last_became_peered":"2024-05-27T08:35:04.022232+0000","last_unstale":"2024-05-27T08:38:16.320491+0000","last_undegraded":"2024-05-27T08:38:16.320491+0000","last_fullsized":"2024-05-27T08:38:16.320491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:49:57.240621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219808+0000","last_change":"2024-05-27T08:35:07.044727+0000","last_active":"2024-05-27T08:38:16.219808+0000","last_peered":"2024-05-27T08:38:16.219808+0000","last_clean":"2024-05-27T08:38:16.219808+0000","last_became_active":"2024-05-27T08:35:07.044352+0000","last_became_peered":"2024-05-27T08:35:07.044352+0000","last_unstale":"2024-05-27T08:38:16.219808+0000","last_undegraded":"2024-05-27T08:38:16.219808+0000","last_fullsized":"2024-05-27T08:38:16.219808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:52:43.473606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"83'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439227+0000","last_change":"2024-05-27T08:35:58.382377+0000","last_active":"2024-05-27T08:38:16.439227+0000","last_peered":"2024-05-27T08:38:16.439227+0000","last_clean":"2024-05-27T08:38:16.439227+0000","last_became_active":"2024-05-27T08:35:58.376093+0000","last_became_peered":"2024-05-27T08:35:58.376093+0000","last_unstale":"2024-05-27T08:38:16.439227+0000","last_undegraded":"2024-05-27T08:38:16.439227+0000","last_fullsized":"2024-05-27T08:38:16.439227+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:00:33.792015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217745+0000","last_change":"2024-05-27T08:35:07.055562+0000","last_active":"2024-05-27T08:38:16.217745+0000","last_peered":"2024-05-27T08:38:16.217745+0000","last_clean":"2024-05-27T08:38:16.217745+0000","last_became_active":"2024-05-27T08:35:07.055209+0000","last_became_peered":"2024-05-27T08:35:07.055209+0000","last_unstale":"2024-05-27T08:38:16.217745+0000","last_undegraded":"2024-05-27T08:38:16.217745+0000","last_fullsized":"2024-05-27T08:38:16.217745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:47:04.920534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'8","reported_seq":219,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:20.526104+0000","last_change":"2024-05-27T08:35:04.022778+0000","last_active":"2024-05-27T08:38:20.526104+0000","last_peered":"2024-05-27T08:38:20.526104+0000","last_clean":"2024-05-27T08:38:20.526104+0000","last_became_active":"2024-05-27T08:35:04.022311+0000","last_became_peered":"2024-05-27T08:35:04.022311+0000","last_unstale":"2024-05-27T08:38:20.526104+0000","last_undegraded":"2024-05-27T08:38:20.526104+0000","last_fullsized":"2024-05-27T08:38:20.526104+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:08:48.174774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320107+0000","last_change":"2024-05-27T08:35:07.040182+0000","last_active":"2024-05-27T08:38:16.320107+0000","last_peered":"2024-05-27T08:38:16.320107+0000","last_clean":"2024-05-27T08:38:16.320107+0000","last_became_active":"2024-05-27T08:35:07.039800+0000","last_became_peered":"2024-05-27T08:35:07.039800+0000","last_unstale":"2024-05-27T08:38:16.320107+0000","last_undegraded":"2024-05-27T08:38:16.320107+0000","last_fullsized":"2024-05-27T08:38:16.320107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:53:53.470284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"83'7","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325736+0000","last_change":"2024-05-27T08:35:04.023757+0000","last_active":"2024-05-27T08:38:16.325736+0000","last_peered":"2024-05-27T08:38:16.325736+0000","last_clean":"2024-05-27T08:38:16.325736+0000","last_became_active":"2024-05-27T08:35:04.023428+0000","last_became_peered":"2024-05-27T08:35:04.023428+0000","last_unstale":"2024-05-27T08:38:16.325736+0000","last_undegraded":"2024-05-27T08:38:16.325736+0000","last_fullsized":"2024-05-27T08:38:16.325736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:58:45.669006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"85'8","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317688+0000","last_change":"2024-05-27T08:35:58.407697+0000","last_active":"2024-05-27T08:38:16.317688+0000","last_peered":"2024-05-27T08:38:16.317688+0000","last_clean":"2024-05-27T08:38:16.317688+0000","last_became_active":"2024-05-27T08:35:58.407379+0000","last_became_peered":"2024-05-27T08:35:58.407379+0000","last_unstale":"2024-05-27T08:38:16.317688+0000","last_undegraded":"2024-05-27T08:38:16.317688+0000","last_fullsized":"2024-05-27T08:38:16.317688+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:01:06.534914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"83'3","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295392+0000","last_change":"2024-05-27T08:35:04.015581+0000","last_active":"2024-05-27T08:38:16.295392+0000","last_peered":"2024-05-27T08:38:16.295392+0000","last_clean":"2024-05-27T08:38:16.295392+0000","last_became_active":"2024-05-27T08:35:04.015215+0000","last_became_peered":"2024-05-27T08:35:04.015215+0000","last_unstale":"2024-05-27T08:38:16.295392+0000","last_undegraded":"2024-05-27T08:38:16.295392+0000","last_fullsized":"2024-05-27T08:38:16.295392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:21:51.942008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218704+0000","last_change":"2024-05-27T08:35:07.049798+0000","last_active":"2024-05-27T08:38:16.218704+0000","last_peered":"2024-05-27T08:38:16.218704+0000","last_clean":"2024-05-27T08:38:16.218704+0000","last_became_active":"2024-05-27T08:35:07.048686+0000","last_became_peered":"2024-05-27T08:35:07.048686+0000","last_unstale":"2024-05-27T08:38:16.218704+0000","last_undegraded":"2024-05-27T08:38:16.218704+0000","last_fullsized":"2024-05-27T08:38:16.218704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:39:23.817581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"83'3","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217470+0000","last_change":"2024-05-27T08:35:04.023826+0000","last_active":"2024-05-27T08:38:16.217470+0000","last_peered":"2024-05-27T08:38:16.217470+0000","last_clean":"2024-05-27T08:38:16.217470+0000","last_became_active":"2024-05-27T08:35:04.023410+0000","last_became_peered":"2024-05-27T08:35:04.023410+0000","last_unstale":"2024-05-27T08:38:16.217470+0000","last_undegraded":"2024-05-27T08:38:16.217470+0000","last_fullsized":"2024-05-27T08:38:16.217470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:38:47.855186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317617+0000","last_change":"2024-05-27T08:35:07.038810+0000","last_active":"2024-05-27T08:38:16.317617+0000","last_peered":"2024-05-27T08:38:16.317617+0000","last_clean":"2024-05-27T08:38:16.317617+0000","last_became_active":"2024-05-27T08:35:07.038480+0000","last_became_peered":"2024-05-27T08:35:07.038480+0000","last_unstale":"2024-05-27T08:38:16.317617+0000","last_undegraded":"2024-05-27T08:38:16.317617+0000","last_fullsized":"2024-05-27T08:38:16.317617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:18:14.429081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'7","reported_seq":392,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:21.526578+0000","last_change":"2024-05-27T08:35:04.026934+0000","last_active":"2024-05-27T08:38:21.526578+0000","last_peered":"2024-05-27T08:38:21.526578+0000","last_clean":"2024-05-27T08:38:21.526578+0000","last_became_active":"2024-05-27T08:35:04.026577+0000","last_became_peered":"2024-05-27T08:35:04.026577+0000","last_unstale":"2024-05-27T08:38:21.526578+0000","last_undegraded":"2024-05-27T08:38:21.526578+0000","last_fullsized":"2024-05-27T08:38:21.526578+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:59:56.067856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":356,"num_read_kb":356,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.220241+0000","last_change":"2024-05-27T08:35:07.043840+0000","last_active":"2024-05-27T08:38:16.220241+0000","last_peered":"2024-05-27T08:38:16.220241+0000","last_clean":"2024-05-27T08:38:16.220241+0000","last_became_active":"2024-05-27T08:35:07.043224+0000","last_became_peered":"2024-05-27T08:35:07.043224+0000","last_unstale":"2024-05-27T08:38:16.220241+0000","last_undegraded":"2024-05-27T08:38:16.220241+0000","last_fullsized":"2024-05-27T08:38:16.220241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:48:20.212038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438880+0000","last_change":"2024-05-27T08:35:04.022602+0000","last_active":"2024-05-27T08:38:16.438880+0000","last_peered":"2024-05-27T08:38:16.438880+0000","last_clean":"2024-05-27T08:38:16.438880+0000","last_became_active":"2024-05-27T08:35:04.022203+0000","last_became_peered":"2024-05-27T08:35:04.022203+0000","last_unstale":"2024-05-27T08:38:16.438880+0000","last_undegraded":"2024-05-27T08:38:16.438880+0000","last_fullsized":"2024-05-27T08:38:16.438880+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:02:51.490696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439428+0000","last_change":"2024-05-27T08:35:07.047257+0000","last_active":"2024-05-27T08:38:16.439428+0000","last_peered":"2024-05-27T08:38:16.439428+0000","last_clean":"2024-05-27T08:38:16.439428+0000","last_became_active":"2024-05-27T08:35:07.046639+0000","last_became_peered":"2024-05-27T08:35:07.046639+0000","last_unstale":"2024-05-27T08:38:16.439428+0000","last_undegraded":"2024-05-27T08:38:16.439428+0000","last_fullsized":"2024-05-27T08:38:16.439428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:04.260111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217915+0000","last_change":"2024-05-27T08:35:04.245295+0000","last_active":"2024-05-27T08:38:16.217915+0000","last_peered":"2024-05-27T08:38:16.217915+0000","last_clean":"2024-05-27T08:38:16.217915+0000","last_became_active":"2024-05-27T08:35:04.244927+0000","last_became_peered":"2024-05-27T08:35:04.244927+0000","last_unstale":"2024-05-27T08:38:16.217915+0000","last_undegraded":"2024-05-27T08:38:16.217915+0000","last_fullsized":"2024-05-27T08:38:16.217915+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:47:58.893480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"85'11","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219915+0000","last_change":"2024-05-27T08:35:07.048872+0000","last_active":"2024-05-27T08:38:16.219915+0000","last_peered":"2024-05-27T08:38:16.219915+0000","last_clean":"2024-05-27T08:38:16.219915+0000","last_became_active":"2024-05-27T08:35:07.048488+0000","last_became_peered":"2024-05-27T08:35:07.048488+0000","last_unstale":"2024-05-27T08:38:16.219915+0000","last_undegraded":"2024-05-27T08:38:16.219915+0000","last_fullsized":"2024-05-27T08:38:16.219915+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:14:31.464923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'39","reported_seq":437,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:21.526407+0000","last_change":"2024-05-27T08:35:04.024718+0000","last_active":"2024-05-27T08:38:21.526407+0000","last_peered":"2024-05-27T08:38:21.526407+0000","last_clean":"2024-05-27T08:38:21.526407+0000","last_became_active":"2024-05-27T08:35:04.024155+0000","last_became_peered":"2024-05-27T08:35:04.024155+0000","last_unstale":"2024-05-27T08:38:21.526407+0000","last_undegraded":"2024-05-27T08:38:21.526407+0000","last_fullsized":"2024-05-27T08:38:21.526407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:35:22.573337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":369,"num_read_kb":249,"num_write":41,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320317+0000","last_change":"2024-05-27T08:35:07.051342+0000","last_active":"2024-05-27T08:38:16.320317+0000","last_peered":"2024-05-27T08:38:16.320317+0000","last_clean":"2024-05-27T08:38:16.320317+0000","last_became_active":"2024-05-27T08:35:07.051030+0000","last_became_peered":"2024-05-27T08:35:07.051030+0000","last_unstale":"2024-05-27T08:38:16.320317+0000","last_undegraded":"2024-05-27T08:38:16.320317+0000","last_fullsized":"2024-05-27T08:38:16.320317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:48.245036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"83'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438727+0000","last_change":"2024-05-27T08:35:04.023717+0000","last_active":"2024-05-27T08:38:16.438727+0000","last_peered":"2024-05-27T08:38:16.438727+0000","last_clean":"2024-05-27T08:38:16.438727+0000","last_became_active":"2024-05-27T08:35:04.022017+0000","last_became_peered":"2024-05-27T08:35:04.022017+0000","last_unstale":"2024-05-27T08:38:16.438727+0000","last_undegraded":"2024-05-27T08:38:16.438727+0000","last_fullsized":"2024-05-27T08:38:16.438727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T08:39:55.834307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295399+0000","last_change":"2024-05-27T08:35:07.043980+0000","last_active":"2024-05-27T08:38:16.295399+0000","last_peered":"2024-05-27T08:38:16.295399+0000","last_clean":"2024-05-27T08:38:16.295399+0000","last_became_active":"2024-05-27T08:35:07.043672+0000","last_became_peered":"2024-05-27T08:35:07.043672+0000","last_unstale":"2024-05-27T08:38:16.295399+0000","last_undegraded":"2024-05-27T08:38:16.295399+0000","last_fullsized":"2024-05-27T08:38:16.295399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:03:30.243966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217554+0000","last_change":"2024-05-27T08:35:04.022731+0000","last_active":"2024-05-27T08:38:16.217554+0000","last_peered":"2024-05-27T08:38:16.217554+0000","last_clean":"2024-05-27T08:38:16.217554+0000","last_became_active":"2024-05-27T08:35:04.022349+0000","last_became_peered":"2024-05-27T08:35:04.022349+0000","last_unstale":"2024-05-27T08:38:16.217554+0000","last_undegraded":"2024-05-27T08:38:16.217554+0000","last_fullsized":"2024-05-27T08:38:16.217554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:38.651520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438589+0000","last_change":"2024-05-27T08:35:07.052623+0000","last_active":"2024-05-27T08:38:16.438589+0000","last_peered":"2024-05-27T08:38:16.438589+0000","last_clean":"2024-05-27T08:38:16.438589+0000","last_became_active":"2024-05-27T08:35:07.052313+0000","last_became_peered":"2024-05-27T08:35:07.052313+0000","last_unstale":"2024-05-27T08:38:16.438589+0000","last_undegraded":"2024-05-27T08:38:16.438589+0000","last_fullsized":"2024-05-27T08:38:16.438589+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:36:24.612523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'8","reported_seq":219,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:20.541927+0000","last_change":"2024-05-27T08:35:04.023756+0000","last_active":"2024-05-27T08:38:20.541927+0000","last_peered":"2024-05-27T08:38:20.541927+0000","last_clean":"2024-05-27T08:38:20.541927+0000","last_became_active":"2024-05-27T08:35:04.023401+0000","last_became_peered":"2024-05-27T08:35:04.023401+0000","last_unstale":"2024-05-27T08:38:20.541927+0000","last_undegraded":"2024-05-27T08:38:20.541927+0000","last_fullsized":"2024-05-27T08:38:20.541927+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:03:34.887246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438391+0000","last_change":"2024-05-27T08:35:07.050720+0000","last_active":"2024-05-27T08:38:16.438391+0000","last_peered":"2024-05-27T08:38:16.438391+0000","last_clean":"2024-05-27T08:38:16.438391+0000","last_became_active":"2024-05-27T08:35:07.050135+0000","last_became_peered":"2024-05-27T08:35:07.050135+0000","last_unstale":"2024-05-27T08:38:16.438391+0000","last_undegraded":"2024-05-27T08:38:16.438391+0000","last_fullsized":"2024-05-27T08:38:16.438391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:40.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"83'17","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320069+0000","last_change":"2024-05-27T08:35:58.421002+0000","last_active":"2024-05-27T08:38:16.320069+0000","last_peered":"2024-05-27T08:38:16.320069+0000","last_clean":"2024-05-27T08:38:16.320069+0000","last_became_active":"2024-05-27T08:35:58.408889+0000","last_became_peered":"2024-05-27T08:35:58.408889+0000","last_unstale":"2024-05-27T08:38:16.320069+0000","last_undegraded":"2024-05-27T08:38:16.320069+0000","last_fullsized":"2024-05-27T08:38:16.320069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:00:29.705436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217489+0000","last_change":"2024-05-27T08:35:07.050504+0000","last_active":"2024-05-27T08:38:16.217489+0000","last_peered":"2024-05-27T08:38:16.217489+0000","last_clean":"2024-05-27T08:38:16.217489+0000","last_became_active":"2024-05-27T08:35:07.049368+0000","last_became_peered":"2024-05-27T08:35:07.049368+0000","last_unstale":"2024-05-27T08:38:16.217489+0000","last_undegraded":"2024-05-27T08:38:16.217489+0000","last_fullsized":"2024-05-27T08:38:16.217489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:28:52.996313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'3","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320511+0000","last_change":"2024-05-27T08:35:04.023924+0000","last_active":"2024-05-27T08:38:16.320511+0000","last_peered":"2024-05-27T08:38:16.320511+0000","last_clean":"2024-05-27T08:38:16.320511+0000","last_became_active":"2024-05-27T08:35:04.023548+0000","last_became_peered":"2024-05-27T08:35:04.023548+0000","last_unstale":"2024-05-27T08:38:16.320511+0000","last_undegraded":"2024-05-27T08:38:16.320511+0000","last_fullsized":"2024-05-27T08:38:16.320511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:09:50.441490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"85'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439437+0000","last_change":"2024-05-27T08:35:58.376335+0000","last_active":"2024-05-27T08:38:16.439437+0000","last_peered":"2024-05-27T08:38:16.439437+0000","last_clean":"2024-05-27T08:38:16.439437+0000","last_became_active":"2024-05-27T08:35:58.375926+0000","last_became_peered":"2024-05-27T08:35:58.375926+0000","last_unstale":"2024-05-27T08:38:16.439437+0000","last_undegraded":"2024-05-27T08:38:16.439437+0000","last_fullsized":"2024-05-27T08:38:16.439437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:35:33.892172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"83'1","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.317833+0000","last_change":"2024-05-27T08:35:58.374310+0000","last_active":"2024-05-27T08:38:16.317833+0000","last_peered":"2024-05-27T08:38:16.317833+0000","last_clean":"2024-05-27T08:38:16.317833+0000","last_became_active":"2024-05-27T08:35:58.373929+0000","last_became_peered":"2024-05-27T08:35:58.373929+0000","last_unstale":"2024-05-27T08:38:16.317833+0000","last_undegraded":"2024-05-27T08:38:16.317833+0000","last_fullsized":"2024-05-27T08:38:16.317833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:30:20.901125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325887+0000","last_change":"2024-05-27T08:35:07.039182+0000","last_active":"2024-05-27T08:38:16.325887+0000","last_peered":"2024-05-27T08:38:16.325887+0000","last_clean":"2024-05-27T08:38:16.325887+0000","last_became_active":"2024-05-27T08:35:07.038793+0000","last_became_peered":"2024-05-27T08:35:07.038793+0000","last_unstale":"2024-05-27T08:38:16.325887+0000","last_undegraded":"2024-05-27T08:38:16.325887+0000","last_fullsized":"2024-05-27T08:38:16.325887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:12:03.839681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'7","reported_seq":216,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:20.557227+0000","last_change":"2024-05-27T08:35:04.023194+0000","last_active":"2024-05-27T08:38:20.557227+0000","last_peered":"2024-05-27T08:38:20.557227+0000","last_clean":"2024-05-27T08:38:20.557227+0000","last_became_active":"2024-05-27T08:35:04.022823+0000","last_became_peered":"2024-05-27T08:35:04.022823+0000","last_unstale":"2024-05-27T08:38:20.557227+0000","last_undegraded":"2024-05-27T08:38:20.557227+0000","last_fullsized":"2024-05-27T08:38:20.557227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:44:28.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1287,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"85'7","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.325773+0000","last_change":"2024-05-27T08:35:07.050717+0000","last_active":"2024-05-27T08:38:16.325773+0000","last_peered":"2024-05-27T08:38:16.325773+0000","last_clean":"2024-05-27T08:38:16.325773+0000","last_became_active":"2024-05-27T08:35:07.050413+0000","last_became_peered":"2024-05-27T08:35:07.050413+0000","last_unstale":"2024-05-27T08:38:16.325773+0000","last_undegraded":"2024-05-27T08:38:16.325773+0000","last_fullsized":"2024-05-27T08:38:16.325773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:00:27.693141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217821+0000","last_change":"2024-05-27T08:35:07.053614+0000","last_active":"2024-05-27T08:38:16.217821+0000","last_peered":"2024-05-27T08:38:16.217821+0000","last_clean":"2024-05-27T08:38:16.217821+0000","last_became_active":"2024-05-27T08:35:07.053253+0000","last_became_peered":"2024-05-27T08:35:07.053253+0000","last_unstale":"2024-05-27T08:38:16.217821+0000","last_undegraded":"2024-05-27T08:38:16.217821+0000","last_fullsized":"2024-05-27T08:38:16.217821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:15:08.985984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'7","reported_seq":215,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:20.541502+0000","last_change":"2024-05-27T08:35:04.246500+0000","last_active":"2024-05-27T08:38:20.541502+0000","last_peered":"2024-05-27T08:38:20.541502+0000","last_clean":"2024-05-27T08:38:20.541502+0000","last_became_active":"2024-05-27T08:35:04.244575+0000","last_became_peered":"2024-05-27T08:35:04.244575+0000","last_unstale":"2024-05-27T08:38:20.541502+0000","last_undegraded":"2024-05-27T08:38:20.541502+0000","last_fullsized":"2024-05-27T08:38:20.541502+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:24:10.818057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218987+0000","last_change":"2024-05-27T08:35:07.048488+0000","last_active":"2024-05-27T08:38:16.218987+0000","last_peered":"2024-05-27T08:38:16.218987+0000","last_clean":"2024-05-27T08:38:16.218987+0000","last_became_active":"2024-05-27T08:35:07.048172+0000","last_became_peered":"2024-05-27T08:35:07.048172+0000","last_unstale":"2024-05-27T08:38:16.218987+0000","last_undegraded":"2024-05-27T08:38:16.218987+0000","last_fullsized":"2024-05-27T08:38:16.218987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:44:01.632944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"83'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255015+0000","last_change":"2024-05-27T08:35:04.022807+0000","last_active":"2024-05-27T08:38:16.255015+0000","last_peered":"2024-05-27T08:38:16.255015+0000","last_clean":"2024-05-27T08:38:16.255015+0000","last_became_active":"2024-05-27T08:35:04.021198+0000","last_became_peered":"2024-05-27T08:35:04.021198+0000","last_unstale":"2024-05-27T08:38:16.255015+0000","last_undegraded":"2024-05-27T08:38:16.255015+0000","last_fullsized":"2024-05-27T08:38:16.255015+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:42:10.100534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"86'99","reported_seq":131,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:18.317821+0000","last_change":"2024-05-27T08:33:45.683600+0000","last_active":"2024-05-27T08:38:18.317821+0000","last_peered":"2024-05-27T08:38:18.317821+0000","last_clean":"2024-05-27T08:38:18.317821+0000","last_became_active":"2024-05-27T08:33:45.677989+0000","last_became_peered":"2024-05-27T08:33:45.677989+0000","last_unstale":"2024-05-27T08:38:18.317821+0000","last_undegraded":"2024-05-27T08:38:18.317821+0000","last_fullsized":"2024-05-27T08:38:18.317821+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:30:25.507727+0000","last_clean_scrub_stamp":"2024-05-27T08:30:25.507727+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:48:26.004814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"85'5","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295270+0000","last_change":"2024-05-27T08:35:07.045879+0000","last_active":"2024-05-27T08:38:16.295270+0000","last_peered":"2024-05-27T08:38:16.295270+0000","last_clean":"2024-05-27T08:38:16.295270+0000","last_became_active":"2024-05-27T08:35:07.045522+0000","last_became_peered":"2024-05-27T08:35:07.045522+0000","last_unstale":"2024-05-27T08:38:16.295270+0000","last_undegraded":"2024-05-27T08:38:16.295270+0000","last_fullsized":"2024-05-27T08:38:16.295270+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:21:30.393769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"83'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438619+0000","last_change":"2024-05-27T08:35:04.022419+0000","last_active":"2024-05-27T08:38:16.438619+0000","last_peered":"2024-05-27T08:38:16.438619+0000","last_clean":"2024-05-27T08:38:16.438619+0000","last_became_active":"2024-05-27T08:35:04.022039+0000","last_became_peered":"2024-05-27T08:35:04.022039+0000","last_unstale":"2024-05-27T08:38:16.438619+0000","last_undegraded":"2024-05-27T08:38:16.438619+0000","last_fullsized":"2024-05-27T08:38:16.438619+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:45:31.260596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"83'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320374+0000","last_change":"2024-05-27T08:35:04.245214+0000","last_active":"2024-05-27T08:38:16.320374+0000","last_peered":"2024-05-27T08:38:16.320374+0000","last_clean":"2024-05-27T08:38:16.320374+0000","last_became_active":"2024-05-27T08:35:04.244768+0000","last_became_peered":"2024-05-27T08:35:04.244768+0000","last_unstale":"2024-05-27T08:38:16.320374+0000","last_undegraded":"2024-05-27T08:38:16.320374+0000","last_fullsized":"2024-05-27T08:38:16.320374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:55:34.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295585+0000","last_change":"2024-05-27T08:35:07.042721+0000","last_active":"2024-05-27T08:38:16.295585+0000","last_peered":"2024-05-27T08:38:16.295585+0000","last_clean":"2024-05-27T08:38:16.295585+0000","last_became_active":"2024-05-27T08:35:07.042435+0000","last_became_peered":"2024-05-27T08:35:07.042435+0000","last_unstale":"2024-05-27T08:38:16.295585+0000","last_undegraded":"2024-05-27T08:38:16.295585+0000","last_fullsized":"2024-05-27T08:38:16.295585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:45:49.186753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'105","reported_seq":225,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:18.485461+0000","last_change":"2024-05-27T08:35:04.017995+0000","last_active":"2024-05-27T08:38:18.485461+0000","last_peered":"2024-05-27T08:38:18.485461+0000","last_clean":"2024-05-27T08:38:18.485461+0000","last_became_active":"2024-05-27T08:35:04.017675+0000","last_became_peered":"2024-05-27T08:35:04.017675+0000","last_unstale":"2024-05-27T08:38:18.485461+0000","last_undegraded":"2024-05-27T08:38:18.485461+0000","last_fullsized":"2024-05-27T08:38:18.485461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:23:15.452624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":191,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219103+0000","last_change":"2024-05-27T08:35:07.046553+0000","last_active":"2024-05-27T08:38:16.219103+0000","last_peered":"2024-05-27T08:38:16.219103+0000","last_clean":"2024-05-27T08:38:16.219103+0000","last_became_active":"2024-05-27T08:35:07.046229+0000","last_became_peered":"2024-05-27T08:35:07.046229+0000","last_unstale":"2024-05-27T08:38:16.219103+0000","last_undegraded":"2024-05-27T08:38:16.219103+0000","last_fullsized":"2024-05-27T08:38:16.219103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T14:03:21.024307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"83'7","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.255144+0000","last_change":"2024-05-27T08:35:04.262439+0000","last_active":"2024-05-27T08:38:16.255144+0000","last_peered":"2024-05-27T08:38:16.255144+0000","last_clean":"2024-05-27T08:38:16.255144+0000","last_became_active":"2024-05-27T08:35:04.261958+0000","last_became_peered":"2024-05-27T08:35:04.261958+0000","last_unstale":"2024-05-27T08:38:16.255144+0000","last_undegraded":"2024-05-27T08:38:16.255144+0000","last_fullsized":"2024-05-27T08:38:16.255144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:11:05.763068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439536+0000","last_change":"2024-05-27T08:35:07.042307+0000","last_active":"2024-05-27T08:38:16.439536+0000","last_peered":"2024-05-27T08:38:16.439536+0000","last_clean":"2024-05-27T08:38:16.439536+0000","last_became_active":"2024-05-27T08:35:07.042001+0000","last_became_peered":"2024-05-27T08:35:07.042001+0000","last_unstale":"2024-05-27T08:38:16.439536+0000","last_undegraded":"2024-05-27T08:38:16.439536+0000","last_fullsized":"2024-05-27T08:38:16.439536+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:23:51.030157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"83'2","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219698+0000","last_change":"2024-05-27T08:35:04.245436+0000","last_active":"2024-05-27T08:38:16.219698+0000","last_peered":"2024-05-27T08:38:16.219698+0000","last_clean":"2024-05-27T08:38:16.219698+0000","last_became_active":"2024-05-27T08:35:04.245039+0000","last_became_peered":"2024-05-27T08:35:04.245039+0000","last_unstale":"2024-05-27T08:38:16.219698+0000","last_undegraded":"2024-05-27T08:38:16.219698+0000","last_fullsized":"2024-05-27T08:38:16.219698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:10:11.756864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"85'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320069+0000","last_change":"2024-05-27T08:35:07.051524+0000","last_active":"2024-05-27T08:38:16.320069+0000","last_peered":"2024-05-27T08:38:16.320069+0000","last_clean":"2024-05-27T08:38:16.320069+0000","last_became_active":"2024-05-27T08:35:07.050676+0000","last_became_peered":"2024-05-27T08:35:07.050676+0000","last_unstale":"2024-05-27T08:38:16.320069+0000","last_undegraded":"2024-05-27T08:38:16.320069+0000","last_fullsized":"2024-05-27T08:38:16.320069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:06.818977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"83'7","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218795+0000","last_change":"2024-05-27T08:35:04.022064+0000","last_active":"2024-05-27T08:38:16.218795+0000","last_peered":"2024-05-27T08:38:16.218795+0000","last_clean":"2024-05-27T08:38:16.218795+0000","last_became_active":"2024-05-27T08:35:04.021691+0000","last_became_peered":"2024-05-27T08:35:04.021691+0000","last_unstale":"2024-05-27T08:38:16.218795+0000","last_undegraded":"2024-05-27T08:38:16.218795+0000","last_fullsized":"2024-05-27T08:38:16.218795+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:07:54.557756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439157+0000","last_change":"2024-05-27T08:35:07.049666+0000","last_active":"2024-05-27T08:38:16.439157+0000","last_peered":"2024-05-27T08:38:16.439157+0000","last_clean":"2024-05-27T08:38:16.439157+0000","last_became_active":"2024-05-27T08:35:07.049312+0000","last_became_peered":"2024-05-27T08:35:07.049312+0000","last_unstale":"2024-05-27T08:38:16.439157+0000","last_undegraded":"2024-05-27T08:38:16.439157+0000","last_fullsized":"2024-05-27T08:38:16.439157+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:37:53.704582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'4","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438605+0000","last_change":"2024-05-27T08:35:04.018457+0000","last_active":"2024-05-27T08:38:16.438605+0000","last_peered":"2024-05-27T08:38:16.438605+0000","last_clean":"2024-05-27T08:38:16.438605+0000","last_became_active":"2024-05-27T08:35:04.018008+0000","last_became_peered":"2024-05-27T08:35:04.018008+0000","last_unstale":"2024-05-27T08:38:16.438605+0000","last_undegraded":"2024-05-27T08:38:16.438605+0000","last_fullsized":"2024-05-27T08:38:16.438605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:08:13.228725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219211+0000","last_change":"2024-05-27T08:35:07.047040+0000","last_active":"2024-05-27T08:38:16.219211+0000","last_peered":"2024-05-27T08:38:16.219211+0000","last_clean":"2024-05-27T08:38:16.219211+0000","last_became_active":"2024-05-27T08:35:07.046731+0000","last_became_peered":"2024-05-27T08:35:07.046731+0000","last_unstale":"2024-05-27T08:38:16.219211+0000","last_undegraded":"2024-05-27T08:38:16.219211+0000","last_fullsized":"2024-05-27T08:38:16.219211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:48:25.368349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"83'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.320164+0000","last_change":"2024-05-27T08:35:04.022804+0000","last_active":"2024-05-27T08:38:16.320164+0000","last_peered":"2024-05-27T08:38:16.320164+0000","last_clean":"2024-05-27T08:38:16.320164+0000","last_became_active":"2024-05-27T08:35:04.022458+0000","last_became_peered":"2024-05-27T08:35:04.022458+0000","last_unstale":"2024-05-27T08:38:16.320164+0000","last_undegraded":"2024-05-27T08:38:16.320164+0000","last_fullsized":"2024-05-27T08:38:16.320164+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T18:42:01.029226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"85'5","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438657+0000","last_change":"2024-05-27T08:35:07.049136+0000","last_active":"2024-05-27T08:38:16.438657+0000","last_peered":"2024-05-27T08:38:16.438657+0000","last_clean":"2024-05-27T08:38:16.438657+0000","last_became_active":"2024-05-27T08:35:07.048793+0000","last_became_peered":"2024-05-27T08:35:07.048793+0000","last_unstale":"2024-05-27T08:38:16.438657+0000","last_undegraded":"2024-05-27T08:38:16.438657+0000","last_fullsized":"2024-05-27T08:38:16.438657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:10:21.191743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":215,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:20.525947+0000","last_change":"2024-05-27T08:35:04.262102+0000","last_active":"2024-05-27T08:38:20.525947+0000","last_peered":"2024-05-27T08:38:20.525947+0000","last_clean":"2024-05-27T08:38:20.525947+0000","last_became_active":"2024-05-27T08:35:04.261665+0000","last_became_peered":"2024-05-27T08:35:04.261665+0000","last_unstale":"2024-05-27T08:38:20.525947+0000","last_undegraded":"2024-05-27T08:38:20.525947+0000","last_fullsized":"2024-05-27T08:38:20.525947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:57:57.712197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":181,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438962+0000","last_change":"2024-05-27T08:35:07.051557+0000","last_active":"2024-05-27T08:38:16.438962+0000","last_peered":"2024-05-27T08:38:16.438962+0000","last_clean":"2024-05-27T08:38:16.438962+0000","last_became_active":"2024-05-27T08:35:07.051246+0000","last_became_peered":"2024-05-27T08:35:07.051246+0000","last_unstale":"2024-05-27T08:38:16.438962+0000","last_undegraded":"2024-05-27T08:38:16.438962+0000","last_fullsized":"2024-05-27T08:38:16.438962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T20:08:13.248301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'67","reported_seq":126,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:18.092934+0000","last_change":"2024-05-27T08:35:04.022328+0000","last_active":"2024-05-27T08:38:18.092934+0000","last_peered":"2024-05-27T08:38:18.092934+0000","last_clean":"2024-05-27T08:38:18.092934+0000","last_became_active":"2024-05-27T08:35:04.021711+0000","last_became_peered":"2024-05-27T08:35:04.021711+0000","last_unstale":"2024-05-27T08:38:18.092934+0000","last_undegraded":"2024-05-27T08:38:18.092934+0000","last_fullsized":"2024-05-27T08:38:18.092934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:42:32.841474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438669+0000","last_change":"2024-05-27T08:35:07.056768+0000","last_active":"2024-05-27T08:38:16.438669+0000","last_peered":"2024-05-27T08:38:16.438669+0000","last_clean":"2024-05-27T08:38:16.438669+0000","last_became_active":"2024-05-27T08:35:07.056482+0000","last_became_peered":"2024-05-27T08:35:07.056482+0000","last_unstale":"2024-05-27T08:38:16.438669+0000","last_undegraded":"2024-05-27T08:38:16.438669+0000","last_fullsized":"2024-05-27T08:38:16.438669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T10:10:23.515625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"83'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295576+0000","last_change":"2024-05-27T08:35:04.016613+0000","last_active":"2024-05-27T08:38:16.295576+0000","last_peered":"2024-05-27T08:38:16.295576+0000","last_clean":"2024-05-27T08:38:16.295576+0000","last_became_active":"2024-05-27T08:35:04.016268+0000","last_became_peered":"2024-05-27T08:35:04.016268+0000","last_unstale":"2024-05-27T08:38:16.295576+0000","last_undegraded":"2024-05-27T08:38:16.295576+0000","last_fullsized":"2024-05-27T08:38:16.295576+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:02.999866+0000","last_clean_scrub_stamp":"2024-05-27T08:35:02.999866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:05:11.881335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"85'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438981+0000","last_change":"2024-05-27T08:35:07.051515+0000","last_active":"2024-05-27T08:38:16.438981+0000","last_peered":"2024-05-27T08:38:16.438981+0000","last_clean":"2024-05-27T08:38:16.438981+0000","last_became_active":"2024-05-27T08:35:07.051197+0000","last_became_peered":"2024-05-27T08:35:07.051197+0000","last_unstale":"2024-05-27T08:38:16.438981+0000","last_undegraded":"2024-05-27T08:38:16.438981+0000","last_fullsized":"2024-05-27T08:38:16.438981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:00:49.233474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"85'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439171+0000","last_change":"2024-05-27T08:35:07.051960+0000","last_active":"2024-05-27T08:38:16.439171+0000","last_peered":"2024-05-27T08:38:16.439171+0000","last_clean":"2024-05-27T08:38:16.439171+0000","last_became_active":"2024-05-27T08:35:07.051395+0000","last_became_peered":"2024-05-27T08:35:07.051395+0000","last_unstale":"2024-05-27T08:38:16.439171+0000","last_undegraded":"2024-05-27T08:38:16.439171+0000","last_fullsized":"2024-05-27T08:38:16.439171+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:45:57.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"85'12","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438517+0000","last_change":"2024-05-27T08:35:07.050096+0000","last_active":"2024-05-27T08:38:16.438517+0000","last_peered":"2024-05-27T08:38:16.438517+0000","last_clean":"2024-05-27T08:38:16.438517+0000","last_became_active":"2024-05-27T08:35:07.049755+0000","last_became_peered":"2024-05-27T08:35:07.049755+0000","last_unstale":"2024-05-27T08:38:16.438517+0000","last_undegraded":"2024-05-27T08:38:16.438517+0000","last_fullsized":"2024-05-27T08:38:16.438517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T16:00:39.768128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438481+0000","last_change":"2024-05-27T08:35:07.048281+0000","last_active":"2024-05-27T08:38:16.438481+0000","last_peered":"2024-05-27T08:38:16.438481+0000","last_clean":"2024-05-27T08:38:16.438481+0000","last_became_active":"2024-05-27T08:35:07.047879+0000","last_became_peered":"2024-05-27T08:35:07.047879+0000","last_unstale":"2024-05-27T08:38:16.438481+0000","last_undegraded":"2024-05-27T08:38:16.438481+0000","last_fullsized":"2024-05-27T08:38:16.438481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T17:49:43.631671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"85'7","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438821+0000","last_change":"2024-05-27T08:35:07.050501+0000","last_active":"2024-05-27T08:38:16.438821+0000","last_peered":"2024-05-27T08:38:16.438821+0000","last_clean":"2024-05-27T08:38:16.438821+0000","last_became_active":"2024-05-27T08:35:07.050159+0000","last_became_peered":"2024-05-27T08:35:07.050159+0000","last_unstale":"2024-05-27T08:38:16.438821+0000","last_undegraded":"2024-05-27T08:38:16.438821+0000","last_fullsized":"2024-05-27T08:38:16.438821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:17:23.293019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"85'4","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295514+0000","last_change":"2024-05-27T08:35:58.372446+0000","last_active":"2024-05-27T08:38:16.295514+0000","last_peered":"2024-05-27T08:38:16.295514+0000","last_clean":"2024-05-27T08:38:16.295514+0000","last_became_active":"2024-05-27T08:35:58.372080+0000","last_became_peered":"2024-05-27T08:35:58.372080+0000","last_unstale":"2024-05-27T08:38:16.295514+0000","last_undegraded":"2024-05-27T08:38:16.295514+0000","last_fullsized":"2024-05-27T08:38:16.295514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:01:57.828720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"85'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439444+0000","last_change":"2024-05-27T08:35:07.052382+0000","last_active":"2024-05-27T08:38:16.439444+0000","last_peered":"2024-05-27T08:38:16.439444+0000","last_clean":"2024-05-27T08:38:16.439444+0000","last_became_active":"2024-05-27T08:35:07.052049+0000","last_became_peered":"2024-05-27T08:35:07.052049+0000","last_unstale":"2024-05-27T08:38:16.439444+0000","last_undegraded":"2024-05-27T08:38:16.439444+0000","last_fullsized":"2024-05-27T08:38:16.439444+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:59:08.912872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"85'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.438897+0000","last_change":"2024-05-27T08:35:07.043810+0000","last_active":"2024-05-27T08:38:16.438897+0000","last_peered":"2024-05-27T08:38:16.438897+0000","last_clean":"2024-05-27T08:38:16.438897+0000","last_became_active":"2024-05-27T08:35:07.043449+0000","last_became_peered":"2024-05-27T08:35:07.043449+0000","last_unstale":"2024-05-27T08:38:16.438897+0000","last_undegraded":"2024-05-27T08:38:16.438897+0000","last_fullsized":"2024-05-27T08:38:16.438897+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T11:57:18.089405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"85'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218995+0000","last_change":"2024-05-27T08:35:58.371407+0000","last_active":"2024-05-27T08:38:16.218995+0000","last_peered":"2024-05-27T08:38:16.218995+0000","last_clean":"2024-05-27T08:38:16.218995+0000","last_became_active":"2024-05-27T08:35:58.370963+0000","last_became_peered":"2024-05-27T08:35:58.370963+0000","last_unstale":"2024-05-27T08:38:16.218995+0000","last_undegraded":"2024-05-27T08:38:16.218995+0000","last_fullsized":"2024-05-27T08:38:16.218995+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T13:12:23.458573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"85'6","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.217613+0000","last_change":"2024-05-27T08:35:07.049960+0000","last_active":"2024-05-27T08:38:16.217613+0000","last_peered":"2024-05-27T08:38:16.217613+0000","last_clean":"2024-05-27T08:38:16.217613+0000","last_became_active":"2024-05-27T08:35:07.049595+0000","last_became_peered":"2024-05-27T08:35:07.049595+0000","last_unstale":"2024-05-27T08:38:16.217613+0000","last_undegraded":"2024-05-27T08:38:16.217613+0000","last_fullsized":"2024-05-27T08:38:16.217613+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T15:12:18.182658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"85'8","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.218743+0000","last_change":"2024-05-27T08:35:07.043790+0000","last_active":"2024-05-27T08:38:16.218743+0000","last_peered":"2024-05-27T08:38:16.218743+0000","last_clean":"2024-05-27T08:38:16.218743+0000","last_became_active":"2024-05-27T08:35:07.043476+0000","last_became_peered":"2024-05-27T08:35:07.043476+0000","last_unstale":"2024-05-27T08:38:16.218743+0000","last_undegraded":"2024-05-27T08:38:16.218743+0000","last_fullsized":"2024-05-27T08:38:16.218743+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:56:01.626979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"85'10","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.219848+0000","last_change":"2024-05-27T08:35:07.044928+0000","last_active":"2024-05-27T08:38:16.219848+0000","last_peered":"2024-05-27T08:38:16.219848+0000","last_clean":"2024-05-27T08:38:16.219848+0000","last_became_active":"2024-05-27T08:35:07.044577+0000","last_became_peered":"2024-05-27T08:35:07.044577+0000","last_unstale":"2024-05-27T08:38:16.219848+0000","last_undegraded":"2024-05-27T08:38:16.219848+0000","last_fullsized":"2024-05-27T08:38:16.219848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T09:39:36.560525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"85'11","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.295715+0000","last_change":"2024-05-27T08:35:07.045067+0000","last_active":"2024-05-27T08:38:16.295715+0000","last_peered":"2024-05-27T08:38:16.295715+0000","last_clean":"2024-05-27T08:38:16.295715+0000","last_became_active":"2024-05-27T08:35:07.044759+0000","last_became_peered":"2024-05-27T08:35:07.044759+0000","last_unstale":"2024-05-27T08:38:16.295715+0000","last_undegraded":"2024-05-27T08:38:16.295715+0000","last_fullsized":"2024-05-27T08:38:16.295715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T12:47:08.440135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"85'8","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-27T08:38:16.439347+0000","last_change":"2024-05-27T08:35:07.048186+0000","last_active":"2024-05-27T08:38:16.439347+0000","last_peered":"2024-05-27T08:38:16.439347+0000","last_clean":"2024-05-27T08:38:16.439347+0000","last_became_active":"2024-05-27T08:35:07.047828+0000","last_became_peered":"2024-05-27T08:35:07.047828+0000","last_unstale":"2024-05-27T08:38:16.439347+0000","last_undegraded":"2024-05-27T08:38:16.439347+0000","last_fullsized":"2024-05-27T08:38:16.439347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-27T08:35:06.017566+0000","last_clean_scrub_stamp":"2024-05-27T08:35:06.017566+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-28T19:59:09.928234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797280,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2170,"num_read_kb":1302,"num_write":509,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1366,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2391840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":387,"ondisk_log_size":387,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":458064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743483,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33240,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711912,"statfs":{"total":95995035648,"available":95960997888,"internally_reserved":0,"allocated":1990656,"data_stored":1443749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939712,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32512,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712640,"statfs":{"total":95995035648,"available":95961743360,"internally_reserved":0,"allocated":1368064,"data_stored":850237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135941,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28336,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716816,"statfs":{"total":95995035648,"available":95966019584,"internally_reserved":0,"allocated":1351680,"data_stored":849361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332169,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1568768,"data_stored":1052071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495695,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":958464,"data_stored":457437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":525496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":458468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":525547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117085,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37116,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708036,"statfs":{"total":95995035648,"available":95957028864,"internally_reserved":0,"allocated":1699840,"data_stored":1178011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920856,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":966656,"data_stored":457634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37324,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707828,"statfs":{"total":95995035648,"available":95956815872,"internally_reserved":0,"allocated":1912832,"data_stored":1378359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":397365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-27T08:38:27.087 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:38:27.327 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:27 smithi179 ceph-mon[36481]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-27T08:38:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:27 smithi179 ceph-mon[36481]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:27 smithi077 ceph-mon[32007]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-27T08:38:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:27 smithi077 ceph-mon[32007]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:28.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:27 smithi092 ceph-mon[36049]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 102 KiB/s wr, 56 op/s 2024-05-27T08:38:28.237 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:27 smithi092 ceph-mon[36049]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-27T08:38:28.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 -- 172.21.15.77:0/3031283535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad8108600 msgr2=0x7f3ad81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:28.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 --2- 172.21.15.77:0/3031283535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad8108600 0x7f3ad81089e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009990 tx=0x7f3ac8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:28.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 -- 172.21.15.77:0/3031283535 shutdown_connections 2024-05-27T08:38:28.909 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 --2- 172.21.15.77:0/3031283535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad81028a0 0x7f3ad8110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:28.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 --2- 172.21.15.77:0/3031283535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad8108600 0x7f3ad81089e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:28.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 --2- 172.21.15.77:0/3031283535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8101ee0 0x7f3ad8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:28.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 -- 172.21.15.77:0/3031283535 >> 172.21.15.77:0/3031283535 conn(0x7f3ad8077760 msgr2=0x7f3ad8077b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:28.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 -- 172.21.15.77:0/3031283535 shutdown_connections 2024-05-27T08:38:28.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 -- 172.21.15.77:0/3031283535 wait complete. 2024-05-27T08:38:28.910 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.907+0000 7f3addb95700 1 Processor -- start 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3addb95700 1 -- start start 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3addb95700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8101ee0 0x7f3ad81030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3addb95700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad81028a0 0x7f3ad8103600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3addb95700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8108600 0x7f3ad8103b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3addb95700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad81137f0 con 0x7f3ad81028a0 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3addb95700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8113670 con 0x7f3ad8101ee0 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3addb95700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad8113970 con 0x7f3ad8108600 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad81028a0 0x7f3ad8103600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad81028a0 0x7f3ad8103600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:39824/0 (socket says 172.21.15.77:39824) 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 -- 172.21.15.77:0/3949087196 learned_addr learned my addr 172.21.15.77:0/3949087196 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:28.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad77fe700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8101ee0 0x7f3ad81030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad7fff700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8108600 0x7f3ad8103b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 -- 172.21.15.77:0/3949087196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8108600 msgr2=0x7f3ad8103b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8108600 0x7f3ad8103b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 -- 172.21.15.77:0/3949087196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8101ee0 msgr2=0x7f3ad81030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8101ee0 0x7f3ad81030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 -- 172.21.15.77:0/3949087196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad81a8310 con 0x7f3ad81028a0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.908+0000 7f3ad6ffd700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad81028a0 0x7f3ad8103600 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b530 tx=0x7f3acc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.909+0000 7f3ad4ff9700 1 -- 172.21.15.77:0/3949087196 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc015020 con 0x7f3ad81028a0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.909+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad81a8600 con 0x7f3ad81028a0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.909+0000 7f3ad4ff9700 1 -- 172.21.15.77:0/3949087196 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3acc011ce0 con 0x7f3ad81028a0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.909+0000 7f3ad4ff9700 1 -- 172.21.15.77:0/3949087196 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc010e40 con 0x7f3ad81028a0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.909+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad81a8b40 con 0x7f3ad81028a0 2024-05-27T08:38:28.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.910+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad806a230 con 0x7f3ad81028a0 2024-05-27T08:38:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.911+0000 7f3ad4ff9700 1 -- 172.21.15.77:0/3949087196 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f3acc018020 con 0x7f3ad81028a0 2024-05-27T08:38:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.912+0000 7f3ad4ff9700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3ac00b2a80 0x7f3ac00b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.912+0000 7f3ad77fe700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3ac00b2a80 0x7f3ac00b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.912+0000 7f3ad4ff9700 1 -- 172.21.15.77:0/3949087196 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3acc0e2500 con 0x7f3ad81028a0 2024-05-27T08:38:28.915 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.912+0000 7f3ad77fe700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3ac00b2a80 0x7f3ac00b4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8006010 tx=0x7f3ac8005960 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:28.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:28.915+0000 7f3ad4ff9700 1 -- 172.21.15.77:0/3949087196 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3acc01f020 con 0x7f3ad81028a0 2024-05-27T08:38:29.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.162+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ad81045f0 con 0x7f3ad81028a0 2024-05-27T08:38:29.164 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.162+0000 7f3ad4ff9700 1 -- 172.21.15.77:0/3949087196 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f3acc0a9e90 con 0x7f3ad81028a0 2024-05-27T08:38:29.165 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:38:29.165 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:38:13.957300+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:38:29.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3ac00b2a80 msgr2=0x7f3ac00b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:29.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3ac00b2a80 0x7f3ac00b4f40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8006010 tx=0x7f3ac8005960 comp rx=0 tx=0).stop 2024-05-27T08:38:29.169 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad81028a0 msgr2=0x7f3ad8103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad81028a0 0x7f3ad8103600 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b530 tx=0x7f3acc00b840 comp rx=0 tx=0).stop 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 shutdown_connections 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3ac00b2a80 0x7f3ac00b4f40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ad8108600 0x7f3ad8103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3ad8101ee0 0x7f3ad81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 --2- 172.21.15.77:0/3949087196 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad81028a0 0x7f3ad8103600 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 >> 172.21.15.77:0/3949087196 conn(0x7f3ad8077760 msgr2=0x7f3ad8100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 shutdown_connections 2024-05-27T08:38:29.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:29.165+0000 7f3addb95700 1 -- 172.21.15.77:0/3949087196 wait complete. 2024-05-27T08:38:29.171 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-05-27T08:38:29.790 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:38:30.029 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:30.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:29 smithi179 ceph-mon[36481]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-05-27T08:38:30.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:29 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3949087196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:29 smithi077 ceph-mon[32007]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-05-27T08:38:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:29 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3949087196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:30.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:29 smithi092 ceph-mon[36049]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 50 op/s 2024-05-27T08:38:30.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:29 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3949087196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:31.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 -- 172.21.15.77:0/1380083947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102470 msgr2=0x7f7fdc1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:31.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/1380083947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102470 0x7f7fdc1028f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009990 tx=0x7f7fc4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 -- 172.21.15.77:0/1380083947 shutdown_connections 2024-05-27T08:38:31.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/1380083947 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fdc102e30 0x7f7fdc10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.674 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/1380083947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc1084b0 0x7f7fdc108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/1380083947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102470 0x7f7fdc1028f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 -- 172.21.15.77:0/1380083947 >> 172.21.15.77:0/1380083947 conn(0x7f7fdc079410 msgr2=0x7f7fdc079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 -- 172.21.15.77:0/1380083947 shutdown_connections 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 -- 172.21.15.77:0/1380083947 wait complete. 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.671+0000 7f7fe13d2700 1 Processor -- start 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fe13d2700 1 -- start start 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fe13d2700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fdc102470 0x7f7fdc19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fe13d2700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102e30 0x7f7fdc19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fe13d2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc1084b0 0x7f7fdc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fe13d2700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc1125e0 con 0x7f7fdc102e30 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fe13d2700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc112460 con 0x7f7fdc102470 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fe13d2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc112760 con 0x7f7fdc1084b0 2024-05-27T08:38:31.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fd27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102e30 0x7f7fdc19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fd27fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102e30 0x7f7fdc19fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54866/0 (socket says 172.21.15.77:54866) 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fd27fc700 1 -- 172.21.15.77:0/2496046431 learned_addr learned my addr 172.21.15.77:0/2496046431 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fdaffd700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fdc102470 0x7f7fdc19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fdb7fe700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc1084b0 0x7f7fdc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fd27fc700 1 -- 172.21.15.77:0/2496046431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc1084b0 msgr2=0x7f7fdc199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fd27fc700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc1084b0 0x7f7fdc199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fd27fc700 1 -- 172.21.15.77:0/2496046431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fdc102470 msgr2=0x7f7fdc19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fd27fc700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fdc102470 0x7f7fdc19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fd27fc700 1 -- 172.21.15.77:0/2496046431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc19a1d0 con 0x7f7fdc102e30 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.672+0000 7f7fdb7fe700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc1084b0 0x7f7fdc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.673+0000 7f7fd27fc700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102e30 0x7f7fdc19fdf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000c00 tx=0x7f7fc4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.673+0000 7f7fd8ff9700 1 -- 172.21.15.77:0/2496046431 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4027070 con 0x7f7fdc102e30 2024-05-27T08:38:31.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.673+0000 7f7fd8ff9700 1 -- 172.21.15.77:0/2496046431 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc4022470 con 0x7f7fdc102e30 2024-05-27T08:38:31.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.673+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc19a460 con 0x7f7fdc102e30 2024-05-27T08:38:31.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.673+0000 7f7fd8ff9700 1 -- 172.21.15.77:0/2496046431 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4017650 con 0x7f7fdc102e30 2024-05-27T08:38:31.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.673+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc1a6780 con 0x7f7fdc102e30 2024-05-27T08:38:31.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.674+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fdc06a230 con 0x7f7fdc102e30 2024-05-27T08:38:31.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.675+0000 7f7fd8ff9700 1 -- 172.21.15.77:0/2496046431 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f7fc402a070 con 0x7f7fdc102e30 2024-05-27T08:38:31.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.676+0000 7f7fd8ff9700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7fc80b2a80 0x7f7fc80b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:31.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.676+0000 7f7fdaffd700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7fc80b2a80 0x7f7fc80b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:31.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.676+0000 7f7fd8ff9700 1 -- 172.21.15.77:0/2496046431 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f7fc40e3a90 con 0x7f7fdc102e30 2024-05-27T08:38:31.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.676+0000 7f7fdaffd700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7fc80b2a80 0x7f7fc80b4f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc19abb0 tx=0x7f7fcc009260 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:31.680 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.679+0000 7f7fd8ff9700 1 -- 172.21.15.77:0/2496046431 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7fc4013070 con 0x7f7fdc102e30 2024-05-27T08:38:31.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.930+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fdc02e560 con 0x7f7fdc102e30 2024-05-27T08:38:31.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.931+0000 7f7fd8ff9700 1 -- 172.21.15.77:0/2496046431 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f7fc40ab3a0 con 0x7f7fdc102e30 2024-05-27T08:38:31.933 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:38:31.933 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:38:13.957300+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7fc80b2a80 msgr2=0x7f7fc80b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7fc80b2a80 0x7f7fc80b4f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc19abb0 tx=0x7f7fcc009260 comp rx=0 tx=0).stop 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102e30 msgr2=0x7f7fdc19fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102e30 0x7f7fdc19fdf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4000c00 tx=0x7f7fc4009580 comp rx=0 tx=0).stop 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 shutdown_connections 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7fc80b2a80 0x7f7fc80b4f40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc1084b0 0x7f7fdc199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7fdc102470 0x7f7fdc19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 --2- 172.21.15.77:0/2496046431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fdc102e30 0x7f7fdc19fdf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 >> 172.21.15.77:0/2496046431 conn(0x7f7fdc079410 msgr2=0x7f7fdc06c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 shutdown_connections 2024-05-27T08:38:31.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:31.933+0000 7f7fe13d2700 1 -- 172.21.15.77:0/2496046431 wait complete. 2024-05-27T08:38:31.936 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-05-27T08:38:32.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:31 smithi179 ceph-mon[36481]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-27T08:38:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:31 smithi077 ceph-mon[32007]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-27T08:38:32.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:31 smithi092 ceph-mon[36049]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-27T08:38:32.611 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:38:32.857 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:33.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:32 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2496046431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:33.116 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:32 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2496046431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:32 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2496046431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:34.106 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:33 smithi077 ceph-mon[32007]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-27T08:38:34.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:33 smithi179 ceph-mon[36481]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-27T08:38:34.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:33 smithi092 ceph-mon[36049]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 46 op/s 2024-05-27T08:38:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 -- 172.21.15.77:0/2573207861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80781084b0 msgr2=0x7f8078108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 --2- 172.21.15.77:0/2573207861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80781084b0 0x7f8078108890 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8068009990 tx=0x7f8068009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 -- 172.21.15.77:0/2573207861 shutdown_connections 2024-05-27T08:38:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 --2- 172.21.15.77:0/2573207861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8078102e30 0x7f807810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 --2- 172.21.15.77:0/2573207861 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f80781084b0 0x7f8078108890 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 --2- 172.21.15.77:0/2573207861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8078102470 0x7f80781028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 -- 172.21.15.77:0/2573207861 >> 172.21.15.77:0/2573207861 conn(0x7f80780793f0 msgr2=0x7f8078079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 -- 172.21.15.77:0/2573207861 shutdown_connections 2024-05-27T08:38:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.500+0000 7f807f500700 1 -- 172.21.15.77:0/2573207861 wait complete. 2024-05-27T08:38:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807f500700 1 Processor -- start 2024-05-27T08:38:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807f500700 1 -- start start 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807f500700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8078102470 0x7f807819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807f500700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8078102e30 0x7f807819b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807f500700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80781084b0 0x7f80781a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807f500700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8078112640 con 0x7f8078102e30 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807f500700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f80781124c0 con 0x7f8078102470 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807f500700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f80781127c0 con 0x7f80781084b0 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807ca9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8078102e30 0x7f807819b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807d29c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8078102470 0x7f807819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807d29c700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8078102470 0x7f807819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:46962/0 (socket says 172.21.15.77:46962) 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807ca9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8078102e30 0x7f807819b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54896/0 (socket says 172.21.15.77:54896) 2024-05-27T08:38:34.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.501+0000 7f807d29c700 1 -- 172.21.15.77:0/3476521149 learned_addr learned my addr 172.21.15.77:0/3476521149 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807da9d700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80781084b0 0x7f80781a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807d29c700 1 -- 172.21.15.77:0/3476521149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80781084b0 msgr2=0x7f80781a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807d29c700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80781084b0 0x7f80781a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807d29c700 1 -- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8078102e30 msgr2=0x7f807819b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807d29c700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8078102e30 0x7f807819b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807d29c700 1 -- 172.21.15.77:0/3476521149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80781a60a0 con 0x7f8078102470 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807ca9b700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8078102e30 0x7f807819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807da9d700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80781084b0 0x7f80781a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807d29c700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8078102470 0x7f807819aae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8068000c00 tx=0x7f80680047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f806e7fc700 1 -- 172.21.15.77:0/3476521149 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068027070 con 0x7f8078102470 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80781988d0 con 0x7f8078102470 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f806e7fc700 1 -- 172.21.15.77:0/3476521149 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8068003f20 con 0x7f8078102470 2024-05-27T08:38:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8078198e00 con 0x7f8078102470 2024-05-27T08:38:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.502+0000 7f806e7fc700 1 -- 172.21.15.77:0/3476521149 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068022c90 con 0x7f8078102470 2024-05-27T08:38:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.503+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f807806a230 con 0x7f8078102470 2024-05-27T08:38:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.505+0000 7f806e7fc700 1 -- 172.21.15.77:0/3476521149 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f806802a070 con 0x7f8078102470 2024-05-27T08:38:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.505+0000 7f806e7fc700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f80640b2a80 0x7f80640b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.506+0000 7f807ca9b700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f80640b2a80 0x7f80640b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.506+0000 7f806e7fc700 1 -- 172.21.15.77:0/3476521149 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f80680e3860 con 0x7f8078102470 2024-05-27T08:38:34.508 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.506+0000 7f807ca9b700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f80640b2a80 0x7f80640b4f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8078079a60 tx=0x7f807400f040 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:34.510 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.509+0000 7f806e7fc700 1 -- 172.21.15.77:0/3476521149 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8068031020 con 0x7f8078102470 2024-05-27T08:38:34.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.717+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f807806c260 con 0x7f8078102470 2024-05-27T08:38:34.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.719+0000 7f806e7fc700 1 -- 172.21.15.77:0/3476521149 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23054 (secure 0 0 0) 0x7f8068013070 con 0x7f8078102470 2024-05-27T08:38:34.720 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:38:34.721 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":86,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:38:16.210241+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/3561834991":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/4211521509":"2024-05-28T08:38:16.210207+0000","172.21.15.77:6801/2118880698":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:6800/2118880698":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/2410459695":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/2484667947":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.721+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f80640b2a80 msgr2=0x7f80640b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.721+0000 7f807f500700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f80640b2a80 0x7f80640b4f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8078079a60 tx=0x7f807400f040 comp rx=0 tx=0).stop 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.721+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8078102470 msgr2=0x7f807819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.721+0000 7f807f500700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8078102470 0x7f807819aae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8068000c00 tx=0x7f80680047b0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.721+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 shutdown_connections 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.721+0000 7f807f500700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f80640b2a80 0x7f80640b4f40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.721+0000 7f807f500700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80781084b0 0x7f80781a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.721+0000 7f807f500700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8078102470 0x7f807819aae0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.722+0000 7f807f500700 1 --2- 172.21.15.77:0/3476521149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8078102e30 0x7f807819b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:34.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.722+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 >> 172.21.15.77:0/3476521149 conn(0x7f80780793f0 msgr2=0x7f8078100580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:34.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.722+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 shutdown_connections 2024-05-27T08:38:34.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:34.722+0000 7f807f500700 1 -- 172.21.15.77:0/3476521149 wait complete. 2024-05-27T08:38:35.032 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:34 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3476521149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:38:35.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:34 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3476521149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:38:35.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:34 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3476521149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:38:35.512 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-27T08:38:35.753 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:36.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:35 smithi179 ceph-mon[36481]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-27T08:38:36.127 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:35 smithi077 ceph-mon[32007]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-27T08:38:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:35 smithi092 ceph-mon[36049]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-05-27T08:38:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:36 smithi077 ceph-mon[32007]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:37.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:36 smithi092 ceph-mon[36049]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:37.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:36 smithi179 ceph-mon[36481]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:38.061 INFO:teuthology.orchestra.run.smithi077.stdout:{"run_dir":"/var/run/ceph"} 2024-05-27T08:38:38.061 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-27T08:38:38.061 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5289 2024-05-27T08:38:38.062 DEBUG:tasks.quiescer:Yielding 2024-05-27T08:38:38.062 INFO:teuthology.run_tasks:Running task workunit... 2024-05-27T08:38:38.076 INFO:tasks.workunit:Pulling workunits from ref a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:38:38.076 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-27T08:38:38.076 INFO:tasks.workunit:timeout=3h 2024-05-27T08:38:38.076 INFO:tasks.workunit:cleanup=True 2024-05-27T08:38:38.076 DEBUG:teuthology.orchestra.run.smithi077:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-27T08:38:38.077 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:38:38.095 INFO:teuthology.orchestra.run.smithi077.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-27T08:38:38.095 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-27T08:38:38.095 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 6bh/107d Inode: 1099511628283 Links: 2 2024-05-27T08:38:38.095 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-27T08:38:38.095 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-27T08:38:38.095 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-05-27 08:35:37.333581856 +0000 2024-05-27T08:38:38.095 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-05-27 08:35:37.333581856 +0000 2024-05-27T08:38:38.095 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-05-27 08:38:13.255186910 +0000 2024-05-27T08:38:38.096 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-05-27 08:35:37.333581856 +0000 2024-05-27T08:38:38.096 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-27T08:38:38.096 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-27T08:38:38.167 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout a9cb3a581a309a99b72997ae5ddb88084f5484c9 2024-05-27T08:38:38.224 INFO:tasks.workunit.client.0.smithi077.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-27T08:38:38.328 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:39.446 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:39 smithi077 ceph-mon[32007]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:38:39.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:39 smithi179 ceph-mon[36481]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:38:39.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:39 smithi092 ceph-mon[36049]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:38:40.029 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 -- 172.21.15.77:0/2368424224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 msgr2=0x7f83941103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 --2- 172.21.15.77:0/2368424224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 0x7f83941103d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f838c00b0a0 tx=0x7f838c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 -- 172.21.15.77:0/2368424224 shutdown_connections 2024-05-27T08:38:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 --2- 172.21.15.77:0/2368424224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 0x7f83941103d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 --2- 172.21.15.77:0/2368424224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83941095b0 0x7f8394109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 --2- 172.21.15.77:0/2368424224 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f839406b220 0x7f839406b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 -- 172.21.15.77:0/2368424224 >> 172.21.15.77:0/2368424224 conn(0x7f83940793f0 msgr2=0x7f8394079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 -- 172.21.15.77:0/2368424224 shutdown_connections 2024-05-27T08:38:40.030 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.026+0000 7f83989c7700 1 -- 172.21.15.77:0/2368424224 wait complete. 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f83989c7700 1 Processor -- start 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f83989c7700 1 -- start start 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f83989c7700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f839406b220 0x7f839419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f83989c7700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 0x7f839419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f83989c7700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83941095b0 0x7f8394199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f83989c7700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83941136e0 con 0x7f839406bbe0 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f83989c7700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f8394113560 con 0x7f839406b220 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f83989c7700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8394113860 con 0x7f83941095b0 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f8389d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 0x7f839419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f8389d9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 0x7f839419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46966/0 (socket says 172.21.15.77:46966) 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.027+0000 7f8389d9b700 1 -- 172.21.15.77:0/2869917796 learned_addr learned my addr 172.21.15.77:0/2869917796 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f8392d9d700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83941095b0 0x7f8394199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:40.031 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f839259c700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f839406b220 0x7f839419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f8389d9b700 1 -- 172.21.15.77:0/2869917796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83941095b0 msgr2=0x7f8394199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f8389d9b700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83941095b0 0x7f8394199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f8389d9b700 1 -- 172.21.15.77:0/2869917796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f839406b220 msgr2=0x7f839419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f8389d9b700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f839406b220 0x7f839419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f8389d9b700 1 -- 172.21.15.77:0/2869917796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839419a1a0 con 0x7f839406bbe0 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f839259c700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f839406b220 0x7f839419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f8389d9b700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 0x7f839419fdc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f837c00d950 tx=0x7f837c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f838bfff700 1 -- 172.21.15.77:0/2869917796 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c016070 con 0x7f839406bbe0 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839419a490 con 0x7f839406bbe0 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.028+0000 7f838bfff700 1 -- 172.21.15.77:0/2869917796 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f837c010460 con 0x7f839406bbe0 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.029+0000 7f838bfff700 1 -- 172.21.15.77:0/2869917796 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c00f620 con 0x7f839406bbe0 2024-05-27T08:38:40.032 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.029+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83941a6750 con 0x7f839406bbe0 2024-05-27T08:38:40.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.030+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839410ba90 con 0x7f839406bbe0 2024-05-27T08:38:40.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.031+0000 7f838bfff700 1 -- 172.21.15.77:0/2869917796 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f837c019070 con 0x7f839406bbe0 2024-05-27T08:38:40.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.031+0000 7f838bfff700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f83800b2a80 0x7f83800b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:40.033 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.031+0000 7f839259c700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f83800b2a80 0x7f83800b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:40.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.031+0000 7f838bfff700 1 -- 172.21.15.77:0/2869917796 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f837c0e2780 con 0x7f839406bbe0 2024-05-27T08:38:40.034 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.032+0000 7f839259c700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f83800b2a80 0x7f83800b4f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f839419ab80 tx=0x7f8384009260 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:40.038 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.036+0000 7f838bfff700 1 -- 172.21.15.77:0/2869917796 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f837c0aa000 con 0x7f839406bbe0 2024-05-27T08:38:40.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.289+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f839419af70 con 0x7f839406bbe0 2024-05-27T08:38:40.291 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.290+0000 7f838bfff700 1 -- 172.21.15.77:0/2869917796 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f837c0a9900 con 0x7f839406bbe0 2024-05-27T08:38:40.292 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:38:40.292 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:38:13.957300+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.292+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f83800b2a80 msgr2=0x7f83800b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.292+0000 7f83989c7700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f83800b2a80 0x7f83800b4f40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f839419ab80 tx=0x7f8384009260 comp rx=0 tx=0).stop 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.292+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 msgr2=0x7f839419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.292+0000 7f83989c7700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 0x7f839419fdc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f837c00d950 tx=0x7f837c00dd10 comp rx=0 tx=0).stop 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.293+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 shutdown_connections 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.293+0000 7f83989c7700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f83800b2a80 0x7f83800b4f40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.293+0000 7f83989c7700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83941095b0 0x7f8394199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.293+0000 7f83989c7700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f839406b220 0x7f839419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.293+0000 7f83989c7700 1 --2- 172.21.15.77:0/2869917796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f839406bbe0 0x7f839419fdc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:40.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.293+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 >> 172.21.15.77:0/2869917796 conn(0x7f83940793f0 msgr2=0x7f8394101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:40.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.293+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 shutdown_connections 2024-05-27T08:38:40.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:40.293+0000 7f83989c7700 1 -- 172.21.15.77:0/2869917796 wait complete. 2024-05-27T08:38:40.295 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-05-27T08:38:40.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:40 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2869917796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:40.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:40 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2869917796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:40.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:40 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2869917796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:38:40.849 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-27T08:35:08.767215+0000', 'modified': '2024-05-27T08:38:13.957300+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24434, 'mds_1': 24433, 'mds_2': 24418, 'mds_3': 24412, 'mds_4': 24410}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6839/3869213740', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 3869213740}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 3869213740}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6841/906525687', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6840', 'nonce': 906525687}, {'type': 'v1', 'addr': '172.21.15.77:6841', 'nonce': 906525687}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24365': {'gid': 24365, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6833/1631050328', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 1631050328}, {'type': 'v1', 'addr': '172.21.15.92:6833', 'nonce': 1631050328}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6833/1008462834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 1008462834}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 1008462834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.92:6835/4019780450', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6834', 'nonce': 4019780450}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 4019780450}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24410': {'gid': 24410, 'name': 'h', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.92:6837/2920087591', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 2920087591}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 2920087591}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24412': {'gid': 24412, 'name': 'f', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.179:6835/2178851321', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 2178851321}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 2178851321}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.179:6837/2857088493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 2857088493}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 2857088493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.179:6839/2672999131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2672999131}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2672999131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.92:6839/709031605', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6838', 'nonce': 709031605}, {'type': 'v1', 'addr': '172.21.15.92:6839', 'nonce': 709031605}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24434, 'qdb_cluster': [24434, 24418, 24410, 24433, 24412]} 2024-05-27T08:38:40.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-27T08:38:40.850 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-05-27T08:38:41.128 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:41.557 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:41 smithi077 ceph-mon[32007]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:41.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:41 smithi179 ceph-mon[36481]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:41.637 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:41 smithi092 ceph-mon[36049]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:42.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 -- 172.21.15.77:0/1257312430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 msgr2=0x7f0204109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 --2- 172.21.15.77:0/1257312430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 0x7f0204109a60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0200009990 tx=0x7f0200009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.707 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 -- 172.21.15.77:0/1257312430 shutdown_connections 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 --2- 172.21.15.77:0/1257312430 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204104220 0x7f0204110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 --2- 172.21.15.77:0/1257312430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 0x7f0204109a60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 --2- 172.21.15.77:0/1257312430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0204103860 0x7f0204103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 -- 172.21.15.77:0/1257312430 >> 172.21.15.77:0/1257312430 conn(0x7f020407b4d0 msgr2=0x7f020407b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 -- 172.21.15.77:0/1257312430 shutdown_connections 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 -- 172.21.15.77:0/1257312430 wait complete. 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 Processor -- start 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.700+0000 7f020c6b8700 1 -- start start 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020c6b8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0204103860 0x7f0204199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020c6b8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204104220 0x7f020419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020c6b8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 0x7f020419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020c6b8700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0204113810 con 0x7f0204109680 2024-05-27T08:38:42.708 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020c6b8700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0204113690 con 0x7f0204104220 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020c6b8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0204113990 con 0x7f0204103860 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 0x7f020419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 0x7f020419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46988/0 (socket says 172.21.15.77:46988) 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 -- 172.21.15.77:0/2033883853 learned_addr learned my addr 172.21.15.77:0/2033883853 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f0209c53700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204104220 0x7f020419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020a454700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0204103860 0x7f0204199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 -- 172.21.15.77:0/2033883853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0204103860 msgr2=0x7f0204199d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0204103860 0x7f0204199d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 -- 172.21.15.77:0/2033883853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204104220 msgr2=0x7f020419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204104220 0x7f020419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 -- 172.21.15.77:0/2033883853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020419ed40 con 0x7f0204109680 2024-05-27T08:38:42.709 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f020ac55700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 0x7f020419e600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f01fc007f10 tx=0x7f01fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.701+0000 7f01fb7fe700 1 -- 172.21.15.77:0/2033883853 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc009e20 con 0x7f0204109680 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.702+0000 7f020c6b8700 1 -- 172.21.15.77:0/2033883853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020419f030 con 0x7f0204109680 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.702+0000 7f01fb7fe700 1 -- 172.21.15.77:0/2033883853 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01fc004bb0 con 0x7f0204109680 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.702+0000 7f01fb7fe700 1 -- 172.21.15.77:0/2033883853 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0058e0 con 0x7f0204109680 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.702+0000 7f020c6b8700 1 -- 172.21.15.77:0/2033883853 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041a68b0 con 0x7f0204109680 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.704+0000 7f01fb7fe700 1 -- 172.21.15.77:0/2033883853 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f01fc01a020 con 0x7f0204109680 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.705+0000 7f01fb7fe700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f01f00b2a80 0x7f01f00b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.705+0000 7f020a454700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f01f00b2a80 0x7f01f00b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.705+0000 7f01fb7fe700 1 -- 172.21.15.77:0/2033883853 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f01fc0e1e70 con 0x7f0204109680 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.705+0000 7f0216f4eb80 1 Processor -- start 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.705+0000 7f020a454700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f01f00b2a80 0x7f01f00b4f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f020000b5c0 tx=0x7f0200005960 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:42.710 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.705+0000 7f0216f4eb80 1 -- start start 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89978bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f8997c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 0x557f8997d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f898d7c40 con 0x557f89978810 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557f898c77e0 con 0x557f8997cf10 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557f898cb820 con 0x557f899791c0 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89978bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89978bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46994/0 (socket says 172.21.15.77:46994) 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 -- 172.21.15.77:0/2531674333 learned_addr learned my addr 172.21.15.77:0/2531674333 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020ac55700 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 0x557f8997d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f0209c53700 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f8997c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 -- 172.21.15.77:0/2531674333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 msgr2=0x557f8997c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f8997c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 -- 172.21.15.77:0/2531674333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 msgr2=0x557f8997d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 0x557f8997d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 -- 172.21.15.77:0/2531674333 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f89817880 con 0x557f89978810 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f020a454700 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89978bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0200015000 tx=0x7f020001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707bceb4de921af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f01f8ff9700 1 -- 172.21.15.77:0/2531674333 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200034070 con 0x557f89978810 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f01f8ff9700 1 -- 172.21.15.77:0/2531674333 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0200024930 con 0x557f89978810 2024-05-27T08:38:42.714 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.706+0000 7f01f8ff9700 1 -- 172.21.15.77:0/2531674333 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020002f910 con 0x557f89978810 2024-05-27T08:38:42.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 -- 172.21.15.77:0/2531674333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 msgr2=0x557f89978bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89978bf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0200015000 tx=0x7f020001f000 comp rx=0 tx=0).stop 2024-05-27T08:38:42.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 -- 172.21.15.77:0/2531674333 shutdown_connections 2024-05-27T08:38:42.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 0x557f8997d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.718 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89978bf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/2531674333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f8997c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 -- 172.21.15.77:0/2531674333 >> 172.21.15.77:0/2531674333 conn(0x557f898a6030 msgr2=0x557f89972e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 -- 172.21.15.77:0/2531674333 shutdown_connections 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.707+0000 7f0216f4eb80 1 -- 172.21.15.77:0/2531674333 wait complete. 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 1 -- start start 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 1 Processor -- start 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89984320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89981620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 0x557f89981b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f897cdf50 con 0x557f89978810 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557f89869770 con 0x557f8997cf10 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557f89985070 con 0x557f899791c0 2024-05-27T08:38:42.719 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0216f4eb80 10 client.0 _ll_register_callbacks cb 0x557f898628d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020a454700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89984320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020a454700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89984320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47006/0 (socket says 172.21.15.77:47006) 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020a454700 1 -- 172.21.15.77:0/479109477 learned_addr learned my addr 172.21.15.77:0/479109477 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020ac55700 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 0x557f89981b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0209c53700 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89981620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020a454700 1 -- 172.21.15.77:0/479109477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 msgr2=0x557f89981620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020a454700 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89981620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020a454700 1 -- 172.21.15.77:0/479109477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 msgr2=0x557f89981b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020a454700 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 0x557f89981b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f020a454700 1 -- 172.21.15.77:0/479109477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f89866640 con 0x557f89978810 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.708+0000 7f0209c53700 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89981620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f020a454700 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89984320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f02000059d0 tx=0x7f0200024d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1afdd9e1b41513f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:42.720 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f01edffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f01edffb700 1 -- 172.21.15.77:0/479109477 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200034040 con 0x557f89978810 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 1 -- 172.21.15.77:0/479109477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0200024930 con 0x557f89978810 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f01edffb700 1 -- 172.21.15.77:0/479109477 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f020003a470 con 0x557f89978810 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f01edffb700 1 -- 172.21.15.77:0/479109477 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f020003a670 con 0x557f89978810 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 10 client.15351 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 1 -- 172.21.15.77:0/479109477 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f8998f730 con 0x557f89978810 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f01edffb700 1 -- 172.21.15.77:0/479109477 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0200024930 con 0x557f89978810 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f01edffb700 1 -- 172.21.15.77:0/479109477 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f020000f9c0 con 0x557f89978810 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 10 client.15351 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 10 client.15351 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 20 client.15351 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x557f89990a80 0x557f89992bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.721 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 4 client.15351 mds_command: new command op to 24410 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.709+0000 7f0216f4eb80 1 -- 172.21.15.77:0/479109477 --> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f89944990 con 0x557f89990a80 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.710+0000 7f0209c53700 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x557f89990a80 0x557f89992bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.710+0000 7f0209c53700 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x557f89990a80 0x557f89992bd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=174a36a9b56d24af server_cookie=51b30d59bd98446b in_seq=0 out_seq=0 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.710+0000 7f01edffb700 10 client.15351 ms_handle_connect on v2:172.21.15.92:6836/2920087591 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f01edffb700 1 -- 172.21.15.77:0/479109477 <== mds.4 v2:172.21.15.92:6836/2920087591 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f89944990 con 0x557f89990a80 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f01edffb700 10 client.15351 handle_command_reply: tid=0 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 client.15351 shutdown 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 -- 172.21.15.77:0/479109477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 msgr2=0x557f89984320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89984320 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f02000059d0 tx=0x7f0200024d10 comp rx=0 tx=0).stop 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 -- 172.21.15.77:0/479109477 shutdown_connections 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f8997cf10 0x557f89981b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x557f89990a80 0x557f89992bd0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89984320 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/479109477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89981620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f01edffb700 0 client.15351 ms_handle_reset on v2:172.21.15.92:6836/2920087591 2024-05-27T08:38:42.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 -- 172.21.15.77:0/479109477 >> 172.21.15.77:0/479109477 conn(0x557f898a6030 msgr2=0x557f899724c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:42.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 -- 172.21.15.77:0/479109477 shutdown_connections 2024-05-27T08:38:42.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 1 -- 172.21.15.77:0/479109477 wait complete. 2024-05-27T08:38:42.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.711+0000 7f0216f4eb80 20 client.15351 trim_cache size 0 max 16384 2024-05-27T08:38:42.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0216f4eb80 1 Processor -- start 2024-05-27T08:38:42.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0216f4eb80 1 -- start start 2024-05-27T08:38:42.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 0x557f8999c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89972c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89973180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.734 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f89990f10 con 0x557f89978810 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557f899915e0 con 0x557f898a6030 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557f89991cb0 con 0x557f899791c0 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0209c53700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89972c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0209c53700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89972c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47014/0 (socket says 172.21.15.77:47014) 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f0209c53700 1 -- 172.21.15.77:0/433033482 learned_addr learned my addr 172.21.15.77:0/433033482 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.732+0000 7f020a454700 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 0x557f8999c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f020ac55700 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89973180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f0209c53700 1 -- 172.21.15.77:0/433033482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 msgr2=0x557f89973180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f0209c53700 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89973180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.735 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f0209c53700 1 -- 172.21.15.77:0/433033482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 msgr2=0x557f8999c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:42.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f0209c53700 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 0x557f8999c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f0209c53700 1 -- 172.21.15.77:0/433033482 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f89981ee0 con 0x557f89978810 2024-05-27T08:38:42.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f020ac55700 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89973180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:42.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f0209c53700 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89972c40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x557f89982bf0 tx=0x7f01f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a0616be639dae3 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:42.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f01effff700 1 -- 172.21.15.77:0/433033482 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4017a80 con 0x557f89978810 2024-05-27T08:38:42.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f01effff700 1 -- 172.21.15.77:0/433033482 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f4017c20 con 0x557f89978810 2024-05-27T08:38:42.736 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.733+0000 7f01effff700 1 -- 172.21.15.77:0/433033482 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f40153c0 con 0x557f89978810 2024-05-27T08:38:42.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 -- 172.21.15.77:0/433033482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 msgr2=0x557f89972c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89972c40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x557f89982bf0 tx=0x7f01f4008040 comp rx=0 tx=0).stop 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 -- 172.21.15.77:0/433033482 shutdown_connections 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f89973180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 0x557f8999c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/433033482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f89972c40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 -- 172.21.15.77:0/433033482 >> 172.21.15.77:0/433033482 conn(0x557f89982510 msgr2=0x557f899836e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 -- 172.21.15.77:0/433033482 shutdown_connections 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 -- 172.21.15.77:0/433033482 wait complete. 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 -- start start 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 Processor -- start 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 0x557f899f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f899f9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.734+0000 7f0216f4eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f899f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557f89992a50 con 0x557f89978810 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557f89992380 con 0x557f898a6030 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0216f4eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557f89993120 con 0x557f899791c0 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f899f9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0216f4eb80 10 client.0 _ll_register_callbacks cb 0x557f898628d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f899f9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47016/0 (socket says 172.21.15.77:47016) 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 -- 172.21.15.77:0/814637340 learned_addr learned my addr 172.21.15.77:0/814637340 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f020a454700 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 0x557f899f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f020ac55700 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f899f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 -- 172.21.15.77:0/814637340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 msgr2=0x557f899f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f899f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 -- 172.21.15.77:0/814637340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 msgr2=0x557f899f97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 0x557f899f97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 -- 172.21.15.77:0/814637340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f899f57c0 con 0x557f89978810 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0209c53700 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f899f9ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006b00 tx=0x7f01f4017b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b27a4a422ee8d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f01edffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f01edffb700 1 -- 172.21.15.77:0/814637340 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f402d070 con 0x557f89978810 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f01edffb700 1 -- 172.21.15.77:0/814637340 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f01f401d520 con 0x557f89978810 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.735+0000 7f0216f4eb80 1 -- 172.21.15.77:0/814637340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f4017c20 con 0x557f89978810 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f01edffb700 1 -- 172.21.15.77:0/814637340 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01f4039050 con 0x557f89978810 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f0216f4eb80 10 client.15357 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f0216f4eb80 1 -- 172.21.15.77:0/814637340 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557f89a02030 con 0x557f89978810 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f01edffb700 1 -- 172.21.15.77:0/814637340 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01f4017c20 con 0x557f89978810 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f01edffb700 1 -- 172.21.15.77:0/814637340 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f01f40153c0 con 0x557f89978810 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f0216f4eb80 10 client.15357 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:42.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f0216f4eb80 10 client.15357 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f0216f4eb80 20 client.15357 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x557f8997cf10 0x557f89a05550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f0216f4eb80 4 client.15357 mds_command: new command op to 24410 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f0216f4eb80 1 -- 172.21.15.77:0/814637340 --> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] -- command(tid 0: {"prefix": "status"}) -- 0x557f898d9b90 con 0x557f8997cf10 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.736+0000 7f020a454700 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x557f8997cf10 0x557f89a05550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f01edffb700 10 client.15357 ms_handle_connect on v2:172.21.15.92:6836/2920087591 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f020a454700 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x557f8997cf10 0x557f89a05550 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ce178dbba06e977 server_cookie=d83fa94b160bbefc in_seq=0 out_seq=0 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f01edffb700 1 -- 172.21.15.77:0/814637340 <== mds.4 v2:172.21.15.92:6836/2920087591 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557f898d9b90 con 0x557f8997cf10 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f01edffb700 10 client.15357 handle_command_reply: tid=0 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 client.15357 shutdown 2024-05-27T08:38:42.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 -- 172.21.15.77:0/814637340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 msgr2=0x557f899f9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f899f9ce0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006b00 tx=0x7f01f4017b00 comp rx=0 tx=0).stop 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 -- 172.21.15.77:0/814637340 shutdown_connections 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557f899791c0 0x557f899f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557f898a6030 0x557f899f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.92:6836/2920087591,v1:172.21.15.92:6837/2920087591] conn(0x557f8997cf10 0x557f89a05550 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 --2- 172.21.15.77:0/814637340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557f89978810 0x557f899f9ce0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 -- 172.21.15.77:0/814637340 >> 172.21.15.77:0/814637340 conn(0x557f89982510 msgr2=0x557f8997fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f01edffb700 0 client.15357 ms_handle_reset on v2:172.21.15.92:6836/2920087591 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 -- 172.21.15.77:0/814637340 shutdown_connections 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.737+0000 7f0216f4eb80 1 -- 172.21.15.77:0/814637340 wait complete. 2024-05-27T08:38:42.745 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f0216f4eb80 20 client.15357 trim_cache size 0 max 16384 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 4, 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24410, 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 199.47371684199999, 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-05-27T08:38:42.746 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 86, 2024-05-27T08:38:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 85, 2024-05-27T08:38:42.747 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 241.96440262900001 2024-05-27T08:38:42.747 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:38:42.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 -- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f01f00b2a80 msgr2=0x7f01f00b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f01f00b2a80 0x7f01f00b4f40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f020000b5c0 tx=0x7f0200005960 comp rx=0 tx=0).stop 2024-05-27T08:38:42.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 -- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 msgr2=0x7f020419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 0x7f020419e600 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f01fc007f10 tx=0x7f01fc00d3b0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 -- 172.21.15.77:0/2033883853 shutdown_connections 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f01f00b2a80 0x7f01f00b4f40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0204109680 0x7f020419e600 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0204103860 0x7f0204199d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 --2- 172.21.15.77:0/2033883853 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0204104220 0x7f020419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 -- 172.21.15.77:0/2033883853 >> 172.21.15.77:0/2033883853 conn(0x7f020407b4d0 msgr2=0x7f0204102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 -- 172.21.15.77:0/2033883853 shutdown_connections 2024-05-27T08:38:42.748 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:42.738+0000 7f020c6b8700 1 -- 172.21.15.77:0/2033883853 wait complete. 2024-05-27T08:38:43.374 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-05-27T08:38:43.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:43 smithi179 ceph-mon[36481]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-27T08:38:43.614 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:43.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:43 smithi077 ceph-mon[32007]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-27T08:38:43.738 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:43 smithi092 ceph-mon[36049]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 9 op/s 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 -- 172.21.15.77:0/2473465101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8102810 msgr2=0x7f1ba8106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 --2- 172.21.15.77:0/2473465101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8102810 0x7f1ba8106810 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1b900099c0 tx=0x7f1b90009cd0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 -- 172.21.15.77:0/2473465101 shutdown_connections 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 --2- 172.21.15.77:0/2473465101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba8106ef0 0x7f1ba810eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 --2- 172.21.15.77:0/2473465101 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8101e60 0x7f1ba8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 --2- 172.21.15.77:0/2473465101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8102810 0x7f1ba8106810 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 -- 172.21.15.77:0/2473465101 >> 172.21.15.77:0/2473465101 conn(0x7f1ba8078ab0 msgr2=0x7f1ba80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 -- 172.21.15.77:0/2473465101 shutdown_connections 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.283+0000 7f1bae87a700 1 -- 172.21.15.77:0/2473465101 wait complete. 2024-05-27T08:38:45.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bae87a700 1 Processor -- start 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bae87a700 1 -- start start 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bae87a700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8101e60 0x7f1ba8199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bae87a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba8102810 0x7f1ba819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bae87a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8106ef0 0x7f1ba819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bae87a700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8111e60 con 0x7f1ba8106ef0 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bae87a700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8111ce0 con 0x7f1ba8101e60 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bae87a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba8111fe0 con 0x7f1ba8102810 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bace17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8106ef0 0x7f1ba819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bace17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8106ef0 0x7f1ba819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47028/0 (socket says 172.21.15.77:47028) 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1bace17700 1 -- 172.21.15.77:0/574702842 learned_addr learned my addr 172.21.15.77:0/574702842 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.284+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8101e60 0x7f1ba8199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba8102810 0x7f1ba819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1bace17700 1 -- 172.21.15.77:0/574702842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba8102810 msgr2=0x7f1ba819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.288 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1bace17700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba8102810 0x7f1ba819a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1bace17700 1 -- 172.21.15.77:0/574702842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8101e60 msgr2=0x7f1ba8199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1bace17700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8101e60 0x7f1ba8199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1bace17700 1 -- 172.21.15.77:0/574702842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba819ed30 con 0x7f1ba8106ef0 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba8102810 0x7f1ba819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1bace17700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8106ef0 0x7f1ba819e5f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000b970 tx=0x7f1ba000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1ba5ffb700 1 -- 172.21.15.77:0/574702842 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0019070 con 0x7f1ba8106ef0 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1bae87a700 1 -- 172.21.15.77:0/574702842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba819f020 con 0x7f1ba8106ef0 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1ba5ffb700 1 -- 172.21.15.77:0/574702842 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ba000cde0 con 0x7f1ba8106ef0 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1ba5ffb700 1 -- 172.21.15.77:0/574702842 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0007370 con 0x7f1ba8106ef0 2024-05-27T08:38:45.289 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.285+0000 7f1bae87a700 1 -- 172.21.15.77:0/574702842 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba81a68a0 con 0x7f1ba8106ef0 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.287+0000 7f1ba5ffb700 1 -- 172.21.15.77:0/574702842 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f1ba001c070 con 0x7f1ba8106ef0 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.288+0000 7f1ba5ffb700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f1b940b2a80 0x7f1b940b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.288+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f1b940b2a80 0x7f1b940b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.288+0000 7f1ba5ffb700 1 -- 172.21.15.77:0/574702842 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1ba00e1ba0 con 0x7f1ba8106ef0 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.288+0000 7f1bb9110b80 1 Processor -- start 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1bb9110b80 1 -- start start 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f1b940b2a80 0x7f1b940b4f40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009a10 tx=0x7f1b98009190 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.292 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1e00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1ec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56307a139d20 con 0x56307a1e5670 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56307a1298c0 con 0x56307a1e0620 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56307a12d900 con 0x56307a1dfc60 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47042/0 (socket says 172.21.15.77:47042) 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 -- 172.21.15.77:0/896009094 learned_addr learned my addr 172.21.15.77:0/896009094 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1e00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1bace17700 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1ec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 -- 172.21.15.77:0/896009094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 msgr2=0x56307a1e00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1e00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 -- 172.21.15.77:0/896009094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 msgr2=0x56307a1ec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1ec410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.293 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 -- 172.21.15.77:0/896009094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307a0d2f60 con 0x56307a1e5670 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.289+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e5a50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800ea00 tx=0x7f1b98011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2158d7b01a0e0e61 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1b9e7fc700 1 -- 172.21.15.77:0/896009094 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9801b040 con 0x56307a1e5670 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1b9e7fc700 1 -- 172.21.15.77:0/896009094 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b98024070 con 0x56307a1e5670 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1b9e7fc700 1 -- 172.21.15.77:0/896009094 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98013740 con 0x56307a1e5670 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 -- 172.21.15.77:0/896009094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 msgr2=0x56307a1e5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e5a50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800ea00 tx=0x7f1b98011000 comp rx=0 tx=0).stop 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 -- 172.21.15.77:0/896009094 shutdown_connections 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1ec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e5a50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/896009094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1e00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 -- 172.21.15.77:0/896009094 >> 172.21.15.77:0/896009094 conn(0x56307a1071f0 msgr2=0x56307a1d6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:45.294 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 -- 172.21.15.77:0/896009094 shutdown_connections 2024-05-27T08:38:45.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.290+0000 7f1bb9110b80 1 -- 172.21.15.77:0/896009094 wait complete. 2024-05-27T08:38:45.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 1 -- start start 2024-05-27T08:38:45.295 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 1 Processor -- start 2024-05-27T08:38:45.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1ec000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1e9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.298 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56307a02ff50 con 0x56307a1e5670 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56307a091d50 con 0x56307a1e0620 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56307a1ecb00 con 0x56307a1dfc60 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.291+0000 7f1bb9110b80 10 client.0 _ll_register_callbacks cb 0x56307a0c4900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47044/0 (socket says 172.21.15.77:47044) 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 -- 172.21.15.77:0/3447777682 learned_addr learned my addr 172.21.15.77:0/3447777682 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1e9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 -- 172.21.15.77:0/3447777682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 msgr2=0x56307a1ec000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1ec000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 -- 172.21.15.77:0/3447777682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 msgr2=0x56307a1e9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1e9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.299 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 -- 172.21.15.77:0/3447777682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307a079880 con 0x56307a1e5670 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1e9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bace17700 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e9840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0000f80 tx=0x7f1ba0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88be082cc51f7a2b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1b877fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1b877fe700 1 -- 172.21.15.77:0/3447777682 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba00ad1e0 con 0x56307a1e5670 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1b877fe700 1 -- 172.21.15.77:0/3447777682 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1ba00b1a10 con 0x56307a1e5670 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bb9110b80 1 -- 172.21.15.77:0/3447777682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b98024070 con 0x56307a1e5670 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1b877fe700 1 -- 172.21.15.77:0/3447777682 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ba00b1c90 con 0x56307a1e5670 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bb9110b80 10 client.15369 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1bb9110b80 1 -- 172.21.15.77:0/3447777682 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56307a0e3710 con 0x56307a1e5670 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.292+0000 7f1b877fe700 1 -- 172.21.15.77:0/3447777682 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b98024070 con 0x56307a1e5670 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1b877fe700 1 -- 172.21.15.77:0/3447777682 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1ba00a7b00 con 0x56307a1e5670 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1bb9110b80 10 client.15369 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1bb9110b80 10 client.15369 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-27T08:38:45.300 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1bb9110b80 20 client.15369 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x56307a1f95e0 0x56307a1faa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1bb9110b80 4 client.15369 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1bb9110b80 1 -- 172.21.15.77:0/3447777682 --> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56307a13bc70 con 0x56307a1f95e0 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x56307a1f95e0 0x56307a1faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1b877fe700 10 client.15369 ms_handle_connect on v2:172.21.15.179:6834/2178851321 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.293+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x56307a1f95e0 0x56307a1faa50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ddf33cd3207343cb server_cookie=51c80f4683a6d85d in_seq=0 out_seq=0 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.294+0000 7f1b877fe700 1 -- 172.21.15.77:0/3447777682 <== mds.3 v2:172.21.15.179:6834/2178851321 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56307a13bc70 con 0x56307a1f95e0 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.294+0000 7f1b877fe700 10 client.15369 handle_command_reply: tid=0 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.294+0000 7f1bb9110b80 1 client.15369 shutdown 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.294+0000 7f1bb9110b80 1 -- 172.21.15.77:0/3447777682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 msgr2=0x56307a1e9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.294+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e9840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0000f80 tx=0x7f1ba0004ad0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.294+0000 7f1bb9110b80 1 -- 172.21.15.77:0/3447777682 shutdown_connections 2024-05-27T08:38:45.301 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.294+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e5670 0x56307a1e9840 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.295+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1ec000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.295+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x56307a1f95e0 0x56307a1faa50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.295+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/3447777682 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1e9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.295+0000 7f1b877fe700 0 client.15369 ms_handle_reset on v2:172.21.15.179:6834/2178851321 2024-05-27T08:38:45.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.295+0000 7f1bb9110b80 1 -- 172.21.15.77:0/3447777682 >> 172.21.15.77:0/3447777682 conn(0x56307a1071f0 msgr2=0x56307a1d5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:45.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.295+0000 7f1bb9110b80 1 -- 172.21.15.77:0/3447777682 shutdown_connections 2024-05-27T08:38:45.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.295+0000 7f1bb9110b80 1 -- 172.21.15.77:0/3447777682 wait complete. 2024-05-27T08:38:45.302 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.295+0000 7f1bb9110b80 20 client.15369 trim_cache size 0 max 16384 2024-05-27T08:38:45.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1bb9110b80 1 Processor -- start 2024-05-27T08:38:45.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1bb9110b80 1 -- start start 2024-05-27T08:38:45.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e0620 0x56307a1e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e5670 0x56307a1da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.317 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56307a1f8d90 con 0x56307a1e0620 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56307a1f9460 con 0x56307a1e5670 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56307a1f9b30 con 0x56307a1dfc60 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1b9ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e0620 0x56307a1e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1b9ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e0620 0x56307a1e4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47052/0 (socket says 172.21.15.77:47052) 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1b9ffff700 1 -- 172.21.15.77:0/4241747406 learned_addr learned my addr 172.21.15.77:0/4241747406 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/4241747406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1b9ffff700 1 -- 172.21.15.77:0/4241747406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 msgr2=0x56307a1e4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/4241747406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1e4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1b9ffff700 1 -- 172.21.15.77:0/4241747406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e5670 msgr2=0x56307a1da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/4241747406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e5670 0x56307a1da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.318 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.316+0000 7f1b9ffff700 1 -- 172.21.15.77:0/4241747406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56307a1f9280 con 0x56307a1e0620 2024-05-27T08:38:45.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/4241747406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e0620 0x56307a1e4b60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1b900099c0 tx=0x7f1b90005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0c5ec284114c32 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:45.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1b9dffb700 1 -- 172.21.15.77:0/4241747406 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90003a60 con 0x56307a1e0620 2024-05-27T08:38:45.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1b9dffb700 1 -- 172.21.15.77:0/4241747406 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b90028070 con 0x56307a1e0620 2024-05-27T08:38:45.321 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1b9dffb700 1 -- 172.21.15.77:0/4241747406 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000baa0 con 0x56307a1e0620 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1bb9110b80 1 -- 172.21.15.77:0/4241747406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e0620 msgr2=0x56307a1e4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/4241747406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e0620 0x56307a1e4b60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1b900099c0 tx=0x7f1b90005fb0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1bb9110b80 1 -- 172.21.15.77:0/4241747406 shutdown_connections 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/4241747406 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e5670 0x56307a1da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/4241747406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1dfc60 0x56307a1e4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/4241747406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1e0620 0x56307a1e4b60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.317+0000 7f1bb9110b80 1 -- 172.21.15.77:0/4241747406 >> 172.21.15.77:0/4241747406 conn(0x56307a1071f0 msgr2=0x56307a1e7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 -- 172.21.15.77:0/4241747406 shutdown_connections 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 -- 172.21.15.77:0/4241747406 wait complete. 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 -- start start 2024-05-27T08:38:45.322 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 Processor -- start 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1dfc60 0x56307a204700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1e5670 0x56307a1d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56307a1fa8d0 con 0x56307a1dfc60 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56307a1fa200 con 0x56307a1e0620 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56307a1fafa0 con 0x56307a1e5670 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.318+0000 7f1bb9110b80 10 client.0 _ll_register_callbacks cb 0x56307a0c4900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1dfc60 0x56307a204700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1dfc60 0x56307a204700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47066/0 (socket says 172.21.15.77:47066) 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 -- 172.21.15.77:0/769132296 learned_addr learned my addr 172.21.15.77:0/769132296 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1bace17700 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1e5670 0x56307a1d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 -- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1e5670 msgr2=0x56307a1d5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1e5670 0x56307a1d5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 -- 172.21.15.77:0/769132296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 msgr2=0x56307a1d5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1d5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 -- 172.21.15.77:0/769132296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56307a1d63d0 con 0x56307a1dfc60 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1bace17700 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1e5670 0x56307a1d5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1ba7fff700 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1dfc60 0x56307a204700 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1b98015f00 tx=0x7f1b98015e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f361d0a6680f4f6a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1b877fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1b877fe700 1 -- 172.21.15.77:0/769132296 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b98035070 con 0x56307a1dfc60 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1b877fe700 1 -- 172.21.15.77:0/769132296 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f1b98015910 con 0x56307a1dfc60 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1bb9110b80 1 -- 172.21.15.77:0/769132296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b90028070 con 0x56307a1dfc60 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.319+0000 7f1b877fe700 1 -- 172.21.15.77:0/769132296 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b98041050 con 0x56307a1dfc60 2024-05-27T08:38:45.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1bb9110b80 10 client.15375 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1bb9110b80 1 -- 172.21.15.77:0/769132296 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56307a264f30 con 0x56307a1dfc60 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1b877fe700 1 -- 172.21.15.77:0/769132296 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b90028070 con 0x56307a1dfc60 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1b877fe700 1 -- 172.21.15.77:0/769132296 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1b9801e030 con 0x56307a1dfc60 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1bb9110b80 10 client.15375 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1bb9110b80 10 client.15375 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1bb9110b80 20 client.15375 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x56307a269740 0x56307a268450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1bb9110b80 4 client.15375 mds_command: new command op to 24412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1bb9110b80 1 -- 172.21.15.77:0/769132296 --> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] -- command(tid 0: {"prefix": "status"}) -- 0x56307a135450 con 0x56307a269740 2024-05-27T08:38:45.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.320+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x56307a269740 0x56307a268450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:45.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1b877fe700 10 client.15375 ms_handle_connect on v2:172.21.15.179:6834/2178851321 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1b9ffff700 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x56307a269740 0x56307a268450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c40bc9f0b3ca410 server_cookie=d7789059fabb3586 in_seq=0 out_seq=0 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1b877fe700 1 -- 172.21.15.77:0/769132296 <== mds.3 v2:172.21.15.179:6834/2178851321 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56307a135450 con 0x56307a269740 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1b877fe700 10 client.15375 handle_command_reply: tid=0 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 client.15375 shutdown 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 -- 172.21.15.77:0/769132296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1dfc60 msgr2=0x56307a204700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1dfc60 0x56307a204700 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1b98015f00 tx=0x7f1b98015e90 comp rx=0 tx=0).stop 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 -- 172.21.15.77:0/769132296 shutdown_connections 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307a1e5670 0x56307a1d5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56307a1dfc60 0x56307a204700 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.179:6834/2178851321,v1:172.21.15.179:6835/2178851321] conn(0x56307a269740 0x56307a268450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 --2- 172.21.15.77:0/769132296 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56307a1e0620 0x56307a1d5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 -- 172.21.15.77:0/769132296 >> 172.21.15.77:0/769132296 conn(0x56307a1071f0 msgr2=0x56307a1e4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1b877fe700 0 client.15375 ms_handle_reset on v2:172.21.15.179:6834/2178851321 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 -- 172.21.15.77:0/769132296 shutdown_connections 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 1 -- 172.21.15.77:0/769132296 wait complete. 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.321+0000 7f1bb9110b80 20 client.15375 trim_cache size 0 max 16384 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 3, 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24412, 2024-05-27T08:38:45.328 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:38:45.329 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:38:45.329 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:38:45.329 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 203.07288685399999, 2024-05-27T08:38:45.329 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-05-27T08:38:45.329 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 86, 2024-05-27T08:38:45.329 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 85, 2024-05-27T08:38:45.329 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 249.670201112 2024-05-27T08:38:45.329 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 -- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f1b940b2a80 msgr2=0x7f1b940b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f1b940b2a80 0x7f1b940b4f40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009a10 tx=0x7f1b98009190 comp rx=0 tx=0).stop 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 -- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8106ef0 msgr2=0x7f1ba819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8106ef0 0x7f1ba819e5f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000b970 tx=0x7f1ba000bc80 comp rx=0 tx=0).stop 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 -- 172.21.15.77:0/574702842 shutdown_connections 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f1b940b2a80 0x7f1b940b4f40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ba8106ef0 0x7f1ba819e5f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ba8101e60 0x7f1ba8199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 --2- 172.21.15.77:0/574702842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ba8102810 0x7f1ba819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 -- 172.21.15.77:0/574702842 >> 172.21.15.77:0/574702842 conn(0x7f1ba8078ab0 msgr2=0x7f1ba8104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:45.330 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 -- 172.21.15.77:0/574702842 shutdown_connections 2024-05-27T08:38:45.331 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:45.322+0000 7f1bae87a700 1 -- 172.21.15.77:0/574702842 wait complete. 2024-05-27T08:38:45.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:45 smithi179 ceph-mon[36481]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-27T08:38:45.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:45 smithi077 ceph-mon[32007]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-27T08:38:45.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:45 smithi092 ceph-mon[36049]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-27T08:38:46.016 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-05-27T08:38:46.257 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:47.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:47 smithi179 ceph-mon[36481]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:38:47.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:47 smithi077 ceph-mon[32007]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:38:47.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:47 smithi092 ceph-mon[36049]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:38:47.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 -- 172.21.15.77:0/1523550074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 msgr2=0x7faee01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 --2- 172.21.15.77:0/1523550074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 0x7faee01103d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faed800b0a0 tx=0x7faed800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 -- 172.21.15.77:0/1523550074 shutdown_connections 2024-05-27T08:38:47.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 --2- 172.21.15.77:0/1523550074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 0x7faee01103d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 --2- 172.21.15.77:0/1523550074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faee01095b0 0x7faee0109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 --2- 172.21.15.77:0/1523550074 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee006bc90 0x7faee006c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 -- 172.21.15.77:0/1523550074 >> 172.21.15.77:0/1523550074 conn(0x7faee0079410 msgr2=0x7faee0079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:47.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 -- 172.21.15.77:0/1523550074 shutdown_connections 2024-05-27T08:38:47.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.931+0000 7faee51a6700 1 -- 172.21.15.77:0/1523550074 wait complete. 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faee51a6700 1 Processor -- start 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faee51a6700 1 -- start start 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faee51a6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faee006bc90 0x7faee019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faee51a6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 0x7faee019ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faee51a6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee01095b0 0x7faee0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faee51a6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faee01137c0 con 0x7faee006c650 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faee51a6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7faee0113640 con 0x7faee01095b0 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faee51a6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faee0113940 con 0x7faee006bc90 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faede59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 0x7faee019ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faede59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 0x7faee019ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47096/0 (socket says 172.21.15.77:47096) 2024-05-27T08:38:47.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faede59c700 1 -- 172.21.15.77:0/501175809 learned_addr learned my addr 172.21.15.77:0/501175809 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.932+0000 7faedf59e700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee01095b0 0x7faee0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faeded9d700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faee006bc90 0x7faee019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faede59c700 1 -- 172.21.15.77:0/501175809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faee006bc90 msgr2=0x7faee019fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faede59c700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faee006bc90 0x7faee019fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faede59c700 1 -- 172.21.15.77:0/501175809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee01095b0 msgr2=0x7faee0199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faede59c700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee01095b0 0x7faee0199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faede59c700 1 -- 172.21.15.77:0/501175809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee019a400 con 0x7faee006c650 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faede59c700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 0x7faee019ff90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faed000b530 tx=0x7faed000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faed7fff700 1 -- 172.21.15.77:0/501175809 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0015020 con 0x7faee006c650 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faee51a6700 1 -- 172.21.15.77:0/501175809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee019a6f0 con 0x7faee006c650 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faed7fff700 1 -- 172.21.15.77:0/501175809 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed0011ce0 con 0x7faee006c650 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faed7fff700 1 -- 172.21.15.77:0/501175809 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0010dd0 con 0x7faee006c650 2024-05-27T08:38:47.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.933+0000 7faee51a6700 1 -- 172.21.15.77:0/501175809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee0102640 con 0x7faee006c650 2024-05-27T08:38:47.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.935+0000 7faed7fff700 1 -- 172.21.15.77:0/501175809 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7faed0018020 con 0x7faee006c650 2024-05-27T08:38:47.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.936+0000 7faed7fff700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7faecc0b2a80 0x7faecc0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.936+0000 7faeded9d700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7faecc0b2a80 0x7faecc0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.936+0000 7faed7fff700 1 -- 172.21.15.77:0/501175809 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7faed00e1d60 con 0x7faee006c650 2024-05-27T08:38:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeded9d700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7faecc0b2a80 0x7faecc0b4f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faec8009a10 tx=0x7faec8009190 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeefa3cb80 1 Processor -- start 2024-05-27T08:38:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeefa3cb80 1 -- start start 2024-05-27T08:38:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 0x55f5659a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f5659ac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5658fdd20 con 0x55f5659a0210 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f5658ed8c0 con 0x55f5659a52d0 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f5658f1900 con 0x55f56599f850 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faedf59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f5659ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faedf59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f5659ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47100/0 (socket says 172.21.15.77:47100) 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faedf59e700 1 -- 172.21.15.77:0/927349204 learned_addr learned my addr 172.21.15.77:0/927349204 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faeded9d700 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 0x55f5659a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.937+0000 7faede59c700 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faedf59e700 1 -- 172.21.15.77:0/927349204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 msgr2=0x55f56599fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faedf59e700 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faedf59e700 1 -- 172.21.15.77:0/927349204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 msgr2=0x55f5659a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:47.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faedf59e700 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 0x55f5659a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faedf59e700 1 -- 172.21.15.77:0/927349204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f565896f60 con 0x55f5659a0210 2024-05-27T08:38:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faedf59e700 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f5659ac030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faed80093c0 tx=0x7faed80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fc4cb740306cc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faed57fa700 1 -- 172.21.15.77:0/927349204 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed8020070 con 0x55f5659a0210 2024-05-27T08:38:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faed57fa700 1 -- 172.21.15.77:0/927349204 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed8003c00 con 0x55f5659a0210 2024-05-27T08:38:47.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.938+0000 7faed57fa700 1 -- 172.21.15.77:0/927349204 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed80052a0 con 0x55f5659a0210 2024-05-27T08:38:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 -- 172.21.15.77:0/927349204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 msgr2=0x55f5659ac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f5659ac030 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7faed80093c0 tx=0x7faed80093f0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 -- 172.21.15.77:0/927349204 shutdown_connections 2024-05-27T08:38:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f5659ac030 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 0x55f5659a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.944 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/927349204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 -- 172.21.15.77:0/927349204 >> 172.21.15.77:0/927349204 conn(0x55f5658cb1f0 msgr2=0x55f56599a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 -- 172.21.15.77:0/927349204 shutdown_connections 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 -- 172.21.15.77:0/927349204 wait complete. 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 -- start start 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 Processor -- start 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 0x55f5659abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.939+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 0x55f5659a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5657f3f50 con 0x55f5659a52d0 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f565855d50 con 0x55f56599f850 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f5659ac630 con 0x55f5659a0210 2024-05-27T08:38:47.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeefa3cb80 10 client.0 _ll_register_callbacks cb 0x55f565888900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659a94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47108/0 (socket says 172.21.15.77:47108) 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 -- 172.21.15.77:0/3359774576 learned_addr learned my addr 172.21.15.77:0/3359774576 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 -- 172.21.15.77:0/3359774576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 msgr2=0x55f5659a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 0x55f5659a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeded9d700 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 0x55f5659abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 -- 172.21.15.77:0/3359774576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 msgr2=0x55f5659abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 0x55f5659abca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 -- 172.21.15.77:0/3359774576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f56583d880 con 0x55f5659a52d0 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faedf59e700 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659a94e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faed800d010 tx=0x7faed8009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc90b9fd1130e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faec67fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faec67fc700 1 -- 172.21.15.77:0/3359774576 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed802a070 con 0x55f5659a52d0 2024-05-27T08:38:47.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeefa3cb80 1 -- 172.21.15.77:0/3359774576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed8003c00 con 0x55f5659a52d0 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faec67fc700 1 -- 172.21.15.77:0/3359774576 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7faed80052c0 con 0x55f5659a52d0 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faec67fc700 1 -- 172.21.15.77:0/3359774576 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faed8036050 con 0x55f5659a52d0 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeefa3cb80 10 client.15387 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.940+0000 7faeefa3cb80 1 -- 172.21.15.77:0/3359774576 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f5658a7710 con 0x55f5659a52d0 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faec67fc700 1 -- 172.21.15.77:0/3359774576 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed8003c00 con 0x55f5659a52d0 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faec67fc700 1 -- 172.21.15.77:0/3359774576 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7faed8011070 con 0x55f5659a52d0 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faeefa3cb80 10 client.15387 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faeefa3cb80 10 client.15387 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faeefa3cb80 20 client.15387 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x55f5659b7e50 0x55f5659b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faeefa3cb80 4 client.15387 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faeefa3cb80 1 -- 172.21.15.77:0/3359774576 --> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5658ffc70 con 0x55f5659b7e50 2024-05-27T08:38:47.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.941+0000 7faeded9d700 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x55f5659b7e50 0x55f5659b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.942+0000 7faec67fc700 10 client.15387 ms_handle_connect on v2:172.21.15.179:6836/2857088493 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.942+0000 7faeded9d700 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x55f5659b7e50 0x55f5659b9ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca5162fd21317426 server_cookie=b100ab3301cf587e in_seq=0 out_seq=0 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faec67fc700 1 -- 172.21.15.77:0/3359774576 <== mds.2 v2:172.21.15.179:6836/2857088493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f5658ffc70 con 0x55f5659b7e50 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faec67fc700 10 client.15387 handle_command_reply: tid=0 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 client.15387 shutdown 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 -- 172.21.15.77:0/3359774576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 msgr2=0x55f5659a94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659a94e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7faed800d010 tx=0x7faed8009c90 comp rx=0 tx=0).stop 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 -- 172.21.15.77:0/3359774576 shutdown_connections 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659a94e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 0x55f5659abca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x55f5659b7e50 0x55f5659b9ff0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faec67fc700 0 client.15387 ms_handle_reset on v2:172.21.15.179:6836/2857088493 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/3359774576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 0x55f5659a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 -- 172.21.15.77:0/3359774576 >> 172.21.15.77:0/3359774576 conn(0x55f5658cb1f0 msgr2=0x55f565999aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:47.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 -- 172.21.15.77:0/3359774576 shutdown_connections 2024-05-27T08:38:47.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 1 -- 172.21.15.77:0/3359774576 wait complete. 2024-05-27T08:38:47.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.943+0000 7faeefa3cb80 20 client.15387 trim_cache size 0 max 16384 2024-05-27T08:38:47.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.964+0000 7faeefa3cb80 1 Processor -- start 2024-05-27T08:38:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.964+0000 7faeefa3cb80 1 -- start start 2024-05-27T08:38:47.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.964+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.964+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f56599ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.964+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 0x55f56599fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.964+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5659b8330 con 0x55f5659a0210 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.964+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f5659b8a00 con 0x55f5659a52d0 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.964+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f5659b90d0 con 0x55f56599f850 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f56599ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f56599ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47120/0 (socket says 172.21.15.77:47120) 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 -- 172.21.15.77:0/1528844927 learned_addr learned my addr 172.21.15.77:0/1528844927 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faeded9d700 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 -- 172.21.15.77:0/1528844927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 msgr2=0x55f56599ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.970 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faedf59e700 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 0x55f56599fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 -- 172.21.15.77:0/1528844927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 msgr2=0x55f56599fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 0x55f56599fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 -- 172.21.15.77:0/1528844927 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f56599c320 con 0x55f5659a0210 2024-05-27T08:38:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faeded9d700 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faede59c700 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f56599ae80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faed00df5a0 tx=0x7faed00a8b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b0202c43627a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faed4ff9700 1 -- 172.21.15.77:0/1528844927 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed00c3070 con 0x55f5659a0210 2024-05-27T08:38:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faed4ff9700 1 -- 172.21.15.77:0/1528844927 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed00ad3a0 con 0x55f5659a0210 2024-05-27T08:38:47.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.965+0000 7faed4ff9700 1 -- 172.21.15.77:0/1528844927 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed00b1c40 con 0x55f5659a0210 2024-05-27T08:38:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 -- 172.21.15.77:0/1528844927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 msgr2=0x55f56599ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f56599ae80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faed00df5a0 tx=0x7faed00a8b10 comp rx=0 tx=0).stop 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 -- 172.21.15.77:0/1528844927 shutdown_connections 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f5659a52d0 0x55f56599fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f56599f850 0x55f56599ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/1528844927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a0210 0x55f56599ae80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 -- 172.21.15.77:0/1528844927 >> 172.21.15.77:0/1528844927 conn(0x55f5658cb1f0 msgr2=0x55f5659a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 -- 172.21.15.77:0/1528844927 shutdown_connections 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 -- 172.21.15.77:0/1528844927 wait complete. 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 -- start start 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.966+0000 7faeefa3cb80 1 Processor -- start 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 0x55f5659c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 0x55f5659c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faeefa3cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659be620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5659b9e70 con 0x55f5659a52d0 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55f5659b97a0 con 0x55f56599f850 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faeefa3cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f5659ba540 con 0x55f5659a0210 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faeefa3cb80 10 client.0 _ll_register_callbacks cb 0x55f565888900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659be620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659be620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:47124/0 (socket says 172.21.15.77:47124) 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faeded9d700 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 0x55f5659c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 -- 172.21.15.77:0/4024492696 learned_addr learned my addr 172.21.15.77:0/4024492696 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faede59c700 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 0x55f5659c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 -- 172.21.15.77:0/4024492696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 msgr2=0x55f5659c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 0x55f5659c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 -- 172.21.15.77:0/4024492696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 msgr2=0x55f5659c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:47.976 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 0x55f5659c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 -- 172.21.15.77:0/4024492696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5659bee60 con 0x55f5659a52d0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faedf59e700 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659be620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faed80095c0 tx=0x7faed8005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d0ffe5bf5d1601 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faec67fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.967+0000 7faec67fc700 1 -- 172.21.15.77:0/4024492696 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed802a040 con 0x55f5659a52d0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faec67fc700 1 -- 172.21.15.77:0/4024492696 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7faed80244d0 con 0x55f5659a52d0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 1 -- 172.21.15.77:0/4024492696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed00ad3a0 con 0x55f5659a52d0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faec67fc700 1 -- 172.21.15.77:0/4024492696 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faed8020040 con 0x55f5659a52d0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 10 client.15393 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 1 -- 172.21.15.77:0/4024492696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f565a25030 con 0x55f5659a52d0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faec67fc700 1 -- 172.21.15.77:0/4024492696 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faed00ad3a0 con 0x55f5659a52d0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faec67fc700 1 -- 172.21.15.77:0/4024492696 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7faed8011050 con 0x55f5659a52d0 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 10 client.15393 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 10 client.15393 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-27T08:38:47.977 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 20 client.15393 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x55f565a29b10 0x55f565a28820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 4 client.15393 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeefa3cb80 1 -- 172.21.15.77:0/4024492696 --> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] -- command(tid 0: {"prefix": "status"}) -- 0x55f5658f9450 con 0x55f565a29b10 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.968+0000 7faeded9d700 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x55f565a29b10 0x55f565a28820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeded9d700 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x55f565a29b10 0x55f565a28820 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=543db0d56b53bb39 server_cookie=c6a993fcb3c258c1 in_seq=0 out_seq=0 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faec67fc700 10 client.15393 ms_handle_connect on v2:172.21.15.179:6836/2857088493 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faec67fc700 1 -- 172.21.15.77:0/4024492696 <== mds.2 v2:172.21.15.179:6836/2857088493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55f5658f9450 con 0x55f565a29b10 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faec67fc700 10 client.15393 handle_command_reply: tid=0 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 client.15393 shutdown 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 -- 172.21.15.77:0/4024492696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 msgr2=0x55f5659be620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659be620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faed80095c0 tx=0x7faed8005e90 comp rx=0 tx=0).stop 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 -- 172.21.15.77:0/4024492696 shutdown_connections 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5659a52d0 0x55f5659be620 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55f56599f850 0x55f5659c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.179:6836/2857088493,v1:172.21.15.179:6837/2857088493] conn(0x55f565a29b10 0x55f565a28820 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 --2- 172.21.15.77:0/4024492696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5659a0210 0x55f5659c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faec67fc700 0 client.15393 ms_handle_reset on v2:172.21.15.179:6836/2857088493 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 -- 172.21.15.77:0/4024492696 >> 172.21.15.77:0/4024492696 conn(0x55f5658cb1f0 msgr2=0x55f56599e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 -- 172.21.15.77:0/4024492696 shutdown_connections 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 1 -- 172.21.15.77:0/4024492696 wait complete. 2024-05-27T08:38:47.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.969+0000 7faeefa3cb80 20 client.15393 trim_cache size 0 max 16384 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 2, 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24418, 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 206.725941404, 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 86, 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 85, 2024-05-27T08:38:47.980 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 249.89995682 2024-05-27T08:38:47.981 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:38:47.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 -- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7faecc0b2a80 msgr2=0x7faecc0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7faecc0b2a80 0x7faecc0b4f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faec8009a10 tx=0x7faec8009190 comp rx=0 tx=0).stop 2024-05-27T08:38:47.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 -- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 msgr2=0x7faee019ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:47.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 0x7faee019ff90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7faed000b530 tx=0x7faed000b840 comp rx=0 tx=0).stop 2024-05-27T08:38:47.981 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 -- 172.21.15.77:0/501175809 shutdown_connections 2024-05-27T08:38:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7faecc0b2a80 0x7faecc0b4f40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faee01095b0 0x7faee0199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faee006bc90 0x7faee019fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 --2- 172.21.15.77:0/501175809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faee006c650 0x7faee019ff90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 -- 172.21.15.77:0/501175809 >> 172.21.15.77:0/501175809 conn(0x7faee0079410 msgr2=0x7faee0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 -- 172.21.15.77:0/501175809 shutdown_connections 2024-05-27T08:38:47.982 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:47.970+0000 7faee51a6700 1 -- 172.21.15.77:0/501175809 wait complete. 2024-05-27T08:38:48.628 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-05-27T08:38:48.869 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:49.577 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:49 smithi077 ceph-mon[32007]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-27T08:38:49.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:49 smithi179 ceph-mon[36481]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-27T08:38:49.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:49 smithi092 ceph-mon[36049]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-05-27T08:38:50.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 -- 172.21.15.77:0/3160334455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 msgr2=0x7ff27c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 --2- 172.21.15.77:0/3160334455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 0x7ff27c108900 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009990 tx=0x7ff26c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 -- 172.21.15.77:0/3160334455 shutdown_connections 2024-05-27T08:38:50.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 --2- 172.21.15.77:0/3160334455 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff27c102e00 0x7ff27c10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 --2- 172.21.15.77:0/3160334455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 0x7ff27c108900 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 --2- 172.21.15.77:0/3160334455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff27c102440 0x7ff27c1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 -- 172.21.15.77:0/3160334455 >> 172.21.15.77:0/3160334455 conn(0x7ff27c0793f0 msgr2=0x7ff27c079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 -- 172.21.15.77:0/3160334455 shutdown_connections 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.579+0000 7ff2834f6700 1 -- 172.21.15.77:0/3160334455 wait complete. 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff2834f6700 1 Processor -- start 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff2834f6700 1 -- start start 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff2834f6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff27c102440 0x7ff27c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff2834f6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff27c102e00 0x7ff27c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff2834f6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 0x7ff27c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff2834f6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c1126d0 con 0x7ff27c108520 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff2834f6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c112550 con 0x7ff27c102440 2024-05-27T08:38:50.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff2834f6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c112850 con 0x7ff27c102e00 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.580+0000 7ff281a93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 0x7ff27c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281a93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 0x7ff27c19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50888/0 (socket says 172.21.15.77:50888) 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281a93700 1 -- 172.21.15.77:0/149244028 learned_addr learned my addr 172.21.15.77:0/149244028 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281292700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff27c102440 0x7ff27c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff280a91700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff27c102e00 0x7ff27c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281a93700 1 -- 172.21.15.77:0/149244028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff27c102e00 msgr2=0x7ff27c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281a93700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff27c102e00 0x7ff27c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281a93700 1 -- 172.21.15.77:0/149244028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff27c102440 msgr2=0x7ff27c199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281a93700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff27c102440 0x7ff27c199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281a93700 1 -- 172.21.15.77:0/149244028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c19ed50 con 0x7ff27c108520 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff280a91700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff27c102e00 0x7ff27c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281292700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff27c102440 0x7ff27c199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:50.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff281a93700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 0x7ff27c19e610 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff27400b970 tx=0x7ff27400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff2727fc700 1 -- 172.21.15.77:0/149244028 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274019070 con 0x7ff27c108520 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff2834f6700 1 -- 172.21.15.77:0/149244028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27c19f040 con 0x7ff27c108520 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff2727fc700 1 -- 172.21.15.77:0/149244028 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff274014440 con 0x7ff27c108520 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff2727fc700 1 -- 172.21.15.77:0/149244028 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27400ce60 con 0x7ff27c108520 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.581+0000 7ff2834f6700 1 -- 172.21.15.77:0/149244028 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27c1a68c0 con 0x7ff27c108520 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.583+0000 7ff2727fc700 1 -- 172.21.15.77:0/149244028 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7ff27401c070 con 0x7ff27c108520 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.584+0000 7ff2727fc700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff2680b2a80 0x7ff2680b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.584+0000 7ff281292700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff2680b2a80 0x7ff2680b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.584+0000 7ff2727fc700 1 -- 172.21.15.77:0/149244028 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff2740e1b70 con 0x7ff27c108520 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff281292700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff2680b2a80 0x7ff2680b4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff26c000c00 tx=0x7ff26c005fb0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff28dd8cb80 1 Processor -- start 2024-05-27T08:38:50.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff28dd8cb80 1 -- start start 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 0x55823f6769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f670c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f6716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55823f5cad20 con 0x55823f676f70 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55823f5ba8c0 con 0x55823f671280 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55823f5be900 con 0x55823f6765c0 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff280a91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f670c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff280a91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f670c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50900/0 (socket says 172.21.15.77:50900) 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff280a91700 1 -- 172.21.15.77:0/979900695 learned_addr learned my addr 172.21.15.77:0/979900695 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.585+0000 7ff281a93700 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f6716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff281292700 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 0x55823f6769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff280a91700 1 -- 172.21.15.77:0/979900695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 msgr2=0x55823f6769a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff280a91700 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 0x55823f6769a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff280a91700 1 -- 172.21.15.77:0/979900695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 msgr2=0x55823f6716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff280a91700 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f6716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff280a91700 1 -- 172.21.15.77:0/979900695 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823f563f60 con 0x55823f676f70 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff281292700 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 0x55823f6769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff280a91700 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f670c60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff27800aa70 tx=0x7ff27800ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad36ccdf1cc414f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff2677fe700 1 -- 172.21.15.77:0/979900695 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278015070 con 0x55823f676f70 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff2677fe700 1 -- 172.21.15.77:0/979900695 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2780113f0 con 0x55823f676f70 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.586+0000 7ff2677fe700 1 -- 172.21.15.77:0/979900695 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780185c0 con 0x55823f676f70 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/979900695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 msgr2=0x55823f670c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f670c60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff27800aa70 tx=0x7ff27800ad80 comp rx=0 tx=0).stop 2024-05-27T08:38:50.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/979900695 shutdown_connections 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f6716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 0x55823f6769a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/979900695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f670c60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/979900695 >> 172.21.15.77:0/979900695 conn(0x55823f5981f0 msgr2=0x55823f667390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/979900695 shutdown_connections 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/979900695 wait complete. 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 -- start start 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.587+0000 7ff28dd8cb80 1 Processor -- start 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f67cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f67a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f676f70 0x55823f67a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55823f4c0f50 con 0x55823f6765c0 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55823f522d50 con 0x55823f671280 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55823f67da50 con 0x55823f676f70 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff28dd8cb80 10 client.0 _ll_register_callbacks cb 0x55823f555900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f67a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f67a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50914/0 (socket says 172.21.15.77:50914) 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 -- 172.21.15.77:0/3883850383 learned_addr learned my addr 172.21.15.77:0/3883850383 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff281292700 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f67cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff281a93700 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f676f70 0x55823f67a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 -- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f676f70 msgr2=0x55823f67a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f676f70 0x55823f67a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 -- 172.21.15.77:0/3883850383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 msgr2=0x55823f67cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f67cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 -- 172.21.15.77:0/3883850383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823f50a880 con 0x55823f6765c0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff281a93700 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f676f70 0x55823f67a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff280a91700 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f67a000 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff278011d70 tx=0x7ff278011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20993f73d7c5169f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.588+0000 7ff264ff9700 1 -- 172.21.15.77:0/3883850383 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278015040 con 0x55823f6765c0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/3883850383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2780113f0 con 0x55823f6765c0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff264ff9700 1 -- 172.21.15.77:0/3883850383 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff27800cca0 con 0x55823f6765c0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff264ff9700 1 -- 172.21.15.77:0/3883850383 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff27800cea0 con 0x55823f6765c0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 10 client.15405 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/3883850383 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55823f574710 con 0x55823f6765c0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff264ff9700 1 -- 172.21.15.77:0/3883850383 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2780113f0 con 0x55823f6765c0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff264ff9700 1 -- 172.21.15.77:0/3883850383 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff278025480 con 0x55823f6765c0 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 10 client.15405 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 10 client.15405 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T08:38:50.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 20 client.15405 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55823f68a1d0 0x55823f68b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 4 client.15405 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/3883850383 --> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55823f5ccc70 con 0x55823f68a1d0 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.589+0000 7ff281292700 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55823f68a1d0 0x55823f68b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.590+0000 7ff264ff9700 10 client.15405 ms_handle_connect on v2:172.21.15.179:6838/2672999131 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.590+0000 7ff281292700 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55823f68a1d0 0x55823f68b640 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a56c5922c2f12258 server_cookie=ab485cda0a7121e6 in_seq=0 out_seq=0 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff264ff9700 1 -- 172.21.15.77:0/3883850383 <== mds.1 v2:172.21.15.179:6838/2672999131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55823f5ccc70 con 0x55823f68a1d0 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff264ff9700 10 client.15405 handle_command_reply: tid=0 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 client.15405 shutdown 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/3883850383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 msgr2=0x55823f67a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f67a000 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff278011d70 tx=0x7ff278011bd0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/3883850383 shutdown_connections 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55823f68a1d0 0x55823f68b640 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f676f70 0x55823f67a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff264ff9700 0 client.15405 ms_handle_reset on v2:172.21.15.179:6838/2672999131 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f67cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/3883850383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f67a000 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/3883850383 >> 172.21.15.77:0/3883850383 conn(0x55823f5981f0 msgr2=0x55823f666aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/3883850383 shutdown_connections 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/3883850383 wait complete. 2024-05-27T08:38:50.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.591+0000 7ff28dd8cb80 20 client.15405 trim_cache size 0 max 16384 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff28dd8cb80 1 Processor -- start 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff28dd8cb80 1 -- start start 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f67a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 0x55823f689290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f667220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55823f689980 con 0x55823f676f70 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55823f68a050 con 0x55823f671280 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55823f68a720 con 0x55823f6765c0 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff281a93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f667220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff281a93700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f667220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50918/0 (socket says 172.21.15.77:50918) 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.612+0000 7ff281a93700 1 -- 172.21.15.77:0/2322711433 learned_addr learned my addr 172.21.15.77:0/2322711433 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff281a93700 1 -- 172.21.15.77:0/2322711433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 msgr2=0x55823f689290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff281a93700 1 --2- 172.21.15.77:0/2322711433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 0x55823f689290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff281a93700 1 -- 172.21.15.77:0/2322711433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 msgr2=0x55823f67a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff281a93700 1 --2- 172.21.15.77:0/2322711433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f67a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff281a93700 1 -- 172.21.15.77:0/2322711433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55823f678590 con 0x55823f676f70 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff281a93700 1 --2- 172.21.15.77:0/2322711433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f667220 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff2740a4e80 tx=0x7ff27400e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16780a986c1f7248 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff266ffd700 1 -- 172.21.15.77:0/2322711433 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2740a8e00 con 0x55823f676f70 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff266ffd700 1 -- 172.21.15.77:0/2322711433 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2740ad1b0 con 0x55823f676f70 2024-05-27T08:38:50.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.613+0000 7ff266ffd700 1 -- 172.21.15.77:0/2322711433 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2740a92c0 con 0x55823f676f70 2024-05-27T08:38:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/2322711433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 msgr2=0x55823f667220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/2322711433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f667220 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff2740a4e80 tx=0x7ff27400e040 comp rx=0 tx=0).stop 2024-05-27T08:38:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/2322711433 shutdown_connections 2024-05-27T08:38:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/2322711433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f676f70 0x55823f667220 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/2322711433 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f671280 0x55823f67a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/2322711433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f6765c0 0x55823f689290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/2322711433 >> 172.21.15.77:0/2322711433 conn(0x55823f5981f0 msgr2=0x55823f663710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/2322711433 shutdown_connections 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/2322711433 wait complete. 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 -- start start 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.614+0000 7ff28dd8cb80 1 Processor -- start 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f671280 0x55823f6668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f666e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff28dd8cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f676f70 0x55823f667370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55823f68b4c0 con 0x55823f6765c0 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55823f68adf0 con 0x55823f676f70 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff28dd8cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55823f68bb90 con 0x55823f671280 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff28dd8cb80 10 client.0 _ll_register_callbacks cb 0x55823f555900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:50.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f666e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f666e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50932/0 (socket says 172.21.15.77:50932) 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 -- 172.21.15.77:0/1861372409 learned_addr learned my addr 172.21.15.77:0/1861372409 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff281a93700 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f676f70 0x55823f667370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff281292700 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f671280 0x55823f6668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 -- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f671280 msgr2=0x55823f6668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f671280 0x55823f6668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 -- 172.21.15.77:0/1861372409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f676f70 msgr2=0x55823f667370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f676f70 0x55823f667370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 -- 172.21.15.77:0/1861372409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823f693130 con 0x55823f6765c0 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff281292700 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f671280 0x55823f6668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff280a91700 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f666e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff27800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a10ccb121a5217 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:50.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff264ff9700 1 -- 172.21.15.77:0/1861372409 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278015030 con 0x55823f6765c0 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/1861372409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2740ad1b0 con 0x55823f6765c0 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.615+0000 7ff264ff9700 1 -- 172.21.15.77:0/1861372409 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff27801a450 con 0x55823f6765c0 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff264ff9700 1 -- 172.21.15.77:0/1861372409 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2780113f0 con 0x55823f6765c0 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff28dd8cb80 10 client.15411 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/1861372409 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55823f6f5e00 con 0x55823f6765c0 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff264ff9700 1 -- 172.21.15.77:0/1861372409 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2740ad1b0 con 0x55823f6765c0 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff264ff9700 1 -- 172.21.15.77:0/1861372409 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff278036060 con 0x55823f6765c0 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff28dd8cb80 10 client.15411 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff28dd8cb80 10 client.15411 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff28dd8cb80 20 client.15411 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55823f68a1d0 0x55823f6f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:50.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff28dd8cb80 4 client.15411 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/1861372409 --> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] -- command(tid 0: {"prefix": "status"}) -- 0x55823f5c6450 con 0x55823f68a1d0 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.616+0000 7ff281292700 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55823f68a1d0 0x55823f6f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff264ff9700 10 client.15411 ms_handle_connect on v2:172.21.15.179:6838/2672999131 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff281292700 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55823f68a1d0 0x55823f6f96d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c4192c85911b827 server_cookie=ee0d354d921e67c in_seq=0 out_seq=0 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff264ff9700 1 -- 172.21.15.77:0/1861372409 <== mds.1 v2:172.21.15.179:6838/2672999131 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55823f5c6450 con 0x55823f68a1d0 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff264ff9700 10 client.15411 handle_command_reply: tid=0 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 client.15411 shutdown 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/1861372409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 msgr2=0x55823f666e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f666e30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff27800cd50 comp rx=0 tx=0).stop 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/1861372409 shutdown_connections 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:6838/2672999131,v1:172.21.15.179:6839/2672999131] conn(0x55823f68a1d0 0x55823f6f96d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55823f676f70 0x55823f667370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff264ff9700 0 client.15411 ms_handle_reset on v2:172.21.15.179:6838/2672999131 2024-05-27T08:38:50.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55823f671280 0x55823f6668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 --2- 172.21.15.77:0/1861372409 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55823f6765c0 0x55823f666e30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/1861372409 >> 172.21.15.77:0/1861372409 conn(0x55823f5981f0 msgr2=0x55823f677350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/1861372409 shutdown_connections 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 1 -- 172.21.15.77:0/1861372409 wait complete. 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.617+0000 7ff28dd8cb80 20 client.15411 trim_cache size 0 max 16384 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 1, 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24433, 2024-05-27T08:38:50.624 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:38:50.625 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:38:50.625 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:38:50.625 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 210.38614090600001, 2024-05-27T08:38:50.625 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-05-27T08:38:50.625 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 86, 2024-05-27T08:38:50.625 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 85, 2024-05-27T08:38:50.625 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 247.20292565099999 2024-05-27T08:38:50.625 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 -- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff2680b2a80 msgr2=0x7ff2680b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff2680b2a80 0x7ff2680b4f40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff26c000c00 tx=0x7ff26c005fb0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 -- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 msgr2=0x7ff27c19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 0x7ff27c19e610 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff27400b970 tx=0x7ff27400bd30 comp rx=0 tx=0).stop 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 -- 172.21.15.77:0/149244028 shutdown_connections 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff2680b2a80 0x7ff2680b4f40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff27c108520 0x7ff27c19e610 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff27c102440 0x7ff27c199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 --2- 172.21.15.77:0/149244028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff27c102e00 0x7ff27c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:50.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 -- 172.21.15.77:0/149244028 >> 172.21.15.77:0/149244028 conn(0x7ff27c0793f0 msgr2=0x7ff27c06b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:50.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 -- 172.21.15.77:0/149244028 shutdown_connections 2024-05-27T08:38:50.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:50.618+0000 7ff2834f6700 1 -- 172.21.15.77:0/149244028 wait complete. 2024-05-27T08:38:51.370 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-05-27T08:38:51.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:51 smithi179 ceph-mon[36481]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:38:51.615 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:38:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:51 smithi077 ceph-mon[32007]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:38:51.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:51 smithi092 ceph-mon[36049]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:38:53.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 -- 172.21.15.77:0/3930924426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a8806bc90 msgr2=0x7f3a8806c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 --2- 172.21.15.77:0/3930924426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a8806bc90 0x7f3a8806c110 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009990 tx=0x7f3a78009ca0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 -- 172.21.15.77:0/3930924426 shutdown_connections 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 --2- 172.21.15.77:0/3930924426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a8806c650 0x7f3a881103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 --2- 172.21.15.77:0/3930924426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a881095b0 0x7f3a88109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 --2- 172.21.15.77:0/3930924426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a8806bc90 0x7f3a8806c110 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 -- 172.21.15.77:0/3930924426 >> 172.21.15.77:0/3930924426 conn(0x7f3a88079410 msgr2=0x7f3a88079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 -- 172.21.15.77:0/3930924426 shutdown_connections 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.198+0000 7f3a8f574700 1 -- 172.21.15.77:0/3930924426 wait complete. 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8f574700 1 Processor -- start 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8f574700 1 -- start start 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8f574700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8806bc90 0x7f3a88102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8f574700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a8806c650 0x7f3a88102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8f574700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a881095b0 0x7f3a88102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8f574700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3a88113820 con 0x7f3a881095b0 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8f574700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3a881136a0 con 0x7f3a8806bc90 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8f574700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3a881139a0 con 0x7f3a8806c650 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8db11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a881095b0 0x7f3a88102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8db11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a881095b0 0x7f3a88102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50956/0 (socket says 172.21.15.77:50956) 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8db11700 1 -- 172.21.15.77:0/400862963 learned_addr learned my addr 172.21.15.77:0/400862963 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8d310700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8806bc90 0x7f3a88102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a8806c650 0x7f3a88102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8db11700 1 -- 172.21.15.77:0/400862963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a8806c650 msgr2=0x7f3a88102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8db11700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a8806c650 0x7f3a88102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8db11700 1 -- 172.21.15.77:0/400862963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8806bc90 msgr2=0x7f3a88102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.199+0000 7f3a8db11700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8806bc90 0x7f3a88102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.200+0000 7f3a8db11700 1 -- 172.21.15.77:0/400862963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a881a8370 con 0x7f3a881095b0 2024-05-27T08:38:53.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.200+0000 7f3a8db11700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a881095b0 0x7f3a88102fc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b970 tx=0x7f3a8000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.200+0000 7f3a7e7fc700 1 -- 172.21.15.77:0/400862963 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80019070 con 0x7f3a881095b0 2024-05-27T08:38:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.200+0000 7f3a8f574700 1 -- 172.21.15.77:0/400862963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a881a8660 con 0x7f3a881095b0 2024-05-27T08:38:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.200+0000 7f3a7e7fc700 1 -- 172.21.15.77:0/400862963 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a8000cde0 con 0x7f3a881095b0 2024-05-27T08:38:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.200+0000 7f3a7e7fc700 1 -- 172.21.15.77:0/400862963 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80007370 con 0x7f3a881095b0 2024-05-27T08:38:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.200+0000 7f3a8f574700 1 -- 172.21.15.77:0/400862963 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a881a8ba0 con 0x7f3a881095b0 2024-05-27T08:38:53.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.202+0000 7f3a7e7fc700 1 -- 172.21.15.77:0/400862963 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f3a8001c070 con 0x7f3a881095b0 2024-05-27T08:38:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.203+0000 7f3a7e7fc700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3a740b2a80 0x7f3a740b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.203+0000 7f3a8d310700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3a740b2a80 0x7f3a740b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.203+0000 7f3a7e7fc700 1 -- 172.21.15.77:0/400862963 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3a800e1bb0 con 0x7f3a881095b0 2024-05-27T08:38:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a8d310700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3a740b2a80 0x7f3a740b4f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009a10 tx=0x7f3a84009190 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:53.206 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a99e0ab80 1 Processor -- start 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a99e0ab80 1 -- start start 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa223c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2236cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2237210 0x556aa2243030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556aa2194d20 con 0x556aa2237210 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556aa21848c0 con 0x556aa223c2d0 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556aa2188900 con 0x556aa2236850 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a8db11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2237210 0x556aa2243030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a8db11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2237210 0x556aa2243030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50968/0 (socket says 172.21.15.77:50968) 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a8db11700 1 -- 172.21.15.77:0/235232108 learned_addr learned my addr 172.21.15.77:0/235232108 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a8d310700 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa223c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.204+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2236cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.207 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a8db11700 1 -- 172.21.15.77:0/235232108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 msgr2=0x556aa2236cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a8db11700 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2236cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a8db11700 1 -- 172.21.15.77:0/235232108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 msgr2=0x556aa223c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a8db11700 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa223c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a8db11700 1 -- 172.21.15.77:0/235232108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa212df60 con 0x556aa2237210 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2236cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a8d310700 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa223c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a8db11700 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2237210 0x556aa2243030 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3a800a8b40 tx=0x7f3a800a8950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c7830a71bb99fe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a737fe700 1 -- 172.21.15.77:0/235232108 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800ab030 con 0x556aa2237210 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a737fe700 1 -- 172.21.15.77:0/235232108 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a800a8e40 con 0x556aa2237210 2024-05-27T08:38:53.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.205+0000 7f3a737fe700 1 -- 172.21.15.77:0/235232108 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800a92e0 con 0x556aa2237210 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/235232108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2237210 msgr2=0x556aa2243030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2237210 0x556aa2243030 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3a800a8b40 tx=0x7f3a800a8950 comp rx=0 tx=0).stop 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/235232108 shutdown_connections 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2237210 0x556aa2243030 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa223c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/235232108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2236cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/235232108 >> 172.21.15.77:0/235232108 conn(0x556aa21621f0 msgr2=0x556aa2231390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/235232108 shutdown_connections 2024-05-27T08:38:53.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/235232108 wait complete. 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 -- start start 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.206+0000 7f3a99e0ab80 1 Processor -- start 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa2242d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2237210 0x556aa2240070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa22405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556aa208af50 con 0x556aa2236850 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556aa20ecd50 con 0x556aa223c2d0 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556aa2243690 con 0x556aa2237210 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a99e0ab80 10 client.0 _ll_register_callbacks cb 0x556aa211f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa2242d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa2242d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50974/0 (socket says 172.21.15.77:50974) 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 -- 172.21.15.77:0/2148238569 learned_addr learned my addr 172.21.15.77:0/2148238569 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2237210 0x556aa2240070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8db11700 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa22405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 -- 172.21.15.77:0/2148238569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2237210 msgr2=0x556aa2240070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2237210 0x556aa2240070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 -- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 msgr2=0x556aa22405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa22405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 -- 172.21.15.77:0/2148238569 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa20d4880 con 0x556aa2236850 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8db11700 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa22405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a8d310700 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa2242d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009450 tx=0x7f3a84004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d2b54e69b24f94 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a70ff9700 1 -- 172.21.15.77:0/2148238569 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84025070 con 0x556aa2236850 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a70ff9700 1 -- 172.21.15.77:0/2148238569 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3a84015bd0 con 0x556aa2236850 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.207+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/2148238569 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a800a8e40 con 0x556aa2236850 2024-05-27T08:38:53.214 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a70ff9700 1 -- 172.21.15.77:0/2148238569 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a8401b040 con 0x556aa2236850 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a99e0ab80 10 client.15423 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/2148238569 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556aa213e710 con 0x556aa2236850 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a70ff9700 1 -- 172.21.15.77:0/2148238569 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a800a8e40 con 0x556aa2236850 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a70ff9700 1 -- 172.21.15.77:0/2148238569 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a840065c0 con 0x556aa2236850 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a99e0ab80 10 client.15423 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a99e0ab80 10 client.15423 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a99e0ab80 20 client.15423 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x556aa2250220 0x556aa2251690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a99e0ab80 4 client.15423 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.208+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/2148238569 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aa2196c70 con 0x556aa2250220 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.209+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x556aa2250220 0x556aa2251690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.209+0000 7f3a70ff9700 10 client.15423 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.209+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x556aa2250220 0x556aa2251690 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=260af40c9a0d9b6f server_cookie=223084b9b5f1f07d in_seq=0 out_seq=0 2024-05-27T08:38:53.215 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a70ff9700 1 -- 172.21.15.77:0/2148238569 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556aa2196c70 con 0x556aa2250220 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a70ff9700 10 client.15423 handle_command_reply: tid=0 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 client.15423 shutdown 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/2148238569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 msgr2=0x556aa2242d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa2242d20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009450 tx=0x7f3a84004860 comp rx=0 tx=0).stop 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/2148238569 shutdown_connections 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x556aa2250220 0x556aa2251690 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa223c2d0 0x556aa22405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a70ff9700 0 client.15423 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa2242d20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/2148238569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2237210 0x556aa2240070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/2148238569 >> 172.21.15.77:0/2148238569 conn(0x556aa21621f0 msgr2=0x556aa2230aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/2148238569 shutdown_connections 2024-05-27T08:38:53.216 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/2148238569 wait complete. 2024-05-27T08:38:53.217 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.210+0000 7f3a99e0ab80 20 client.15423 trim_cache size 0 max 16384 2024-05-27T08:38:53.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.231+0000 7f3a99e0ab80 1 Processor -- start 2024-05-27T08:38:53.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.231+0000 7f3a99e0ab80 1 -- start start 2024-05-27T08:38:53.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.231+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2231220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.233 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.231+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa2235f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.231+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa223c2d0 0x556aa223cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.231+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556aa224f9d0 con 0x556aa223c2d0 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.231+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556aa22500a0 con 0x556aa2237210 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.231+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556aa2250770 con 0x556aa2236850 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa223c2d0 0x556aa223cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa223c2d0 0x556aa223cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50982/0 (socket says 172.21.15.77:50982) 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 -- 172.21.15.77:0/3751572953 learned_addr learned my addr 172.21.15.77:0/3751572953 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa2235f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8d310700 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2231220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 -- 172.21.15.77:0/3751572953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 msgr2=0x556aa2231220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2231220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.234 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 -- 172.21.15.77:0/3751572953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 msgr2=0x556aa2235f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:53.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa2235f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 -- 172.21.15.77:0/3751572953 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aa2233330 con 0x556aa223c2d0 2024-05-27T08:38:53.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a8db11700 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa223c2d0 0x556aa223cfc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3a800df440 tx=0x7f3a800a86e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd948e2bf1b50015 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:53.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a72ffd700 1 -- 172.21.15.77:0/3751572953 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800c7020 con 0x556aa223c2d0 2024-05-27T08:38:53.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a72ffd700 1 -- 172.21.15.77:0/3751572953 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a800b1ca0 con 0x556aa223c2d0 2024-05-27T08:38:53.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.232+0000 7f3a72ffd700 1 -- 172.21.15.77:0/3751572953 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800ab040 con 0x556aa223c2d0 2024-05-27T08:38:53.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/3751572953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa223c2d0 msgr2=0x556aa223cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa223c2d0 0x556aa223cfc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3a800df440 tx=0x7f3a800a86e0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/3751572953 shutdown_connections 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa223c2d0 0x556aa223cfc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa2236850 0x556aa2231220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/3751572953 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa2235f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/3751572953 >> 172.21.15.77:0/3751572953 conn(0x556aa21621f0 msgr2=0x556aa2235840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/3751572953 shutdown_connections 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/3751572953 wait complete. 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 -- start start 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.233+0000 7f3a99e0ab80 1 Processor -- start 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa225ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa225b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a99e0ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa223c2d0 0x556aa2253c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556aa2251510 con 0x556aa2236850 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x556aa2250e40 con 0x556aa2237210 2024-05-27T08:38:53.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a99e0ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556aa2251be0 con 0x556aa223c2d0 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a99e0ab80 10 client.0 _ll_register_callbacks cb 0x556aa211f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8d310700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa225ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8cb0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa225b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8d310700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa225ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50984/0 (socket says 172.21.15.77:50984) 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8d310700 1 -- 172.21.15.77:0/450962986 learned_addr learned my addr 172.21.15.77:0/450962986 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8cb0f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa225b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:44596/0 (socket says 172.21.15.77:44596) 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8db11700 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa223c2d0 0x556aa2253c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8d310700 1 -- 172.21.15.77:0/450962986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa223c2d0 msgr2=0x556aa2253c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8d310700 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa223c2d0 0x556aa2253c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8d310700 1 -- 172.21.15.77:0/450962986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 msgr2=0x556aa225b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8d310700 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa225b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8d310700 1 -- 172.21.15.77:0/450962986 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aa22541d0 con 0x556aa2236850 2024-05-27T08:38:53.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.234+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa225b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a8d310700 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa225ab10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3a84000c00 tx=0x7f3a84015e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46241940bf5c9cb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a70ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a70ff9700 1 -- 172.21.15.77:0/450962986 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84025040 con 0x556aa2236850 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a70ff9700 1 -- 172.21.15.77:0/450962986 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3a84021d90 con 0x556aa2236850 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/450962986 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556aa2190450 con 0x556aa2236850 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a70ff9700 1 -- 172.21.15.77:0/450962986 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a84043050 con 0x556aa2236850 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a99e0ab80 10 client.15429 fetch_fsmap learned FSMap version 29 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/450962986 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556aa22bc490 con 0x556aa2236850 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a70ff9700 1 -- 172.21.15.77:0/450962986 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556aa2190450 con 0x556aa2236850 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a70ff9700 1 -- 172.21.15.77:0/450962986 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a8402a440 con 0x556aa2236850 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a99e0ab80 10 client.15429 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.235+0000 7f3a99e0ab80 10 client.15429 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-27T08:38:53.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a99e0ab80 20 client.15429 populate_metadata read hostname 'smithi077' 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x556aa22c1130 0x556aa22bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a99e0ab80 4 client.15429 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/450962986 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "status"}) -- 0x556aa2007200 con 0x556aa22c1130 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x556aa22c1130 0x556aa22bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a8cb0f700 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x556aa22c1130 0x556aa22bfe40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43547cd6b24f252a server_cookie=7f74c327979fac8c in_seq=0 out_seq=0 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a70ff9700 10 client.15429 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a70ff9700 1 -- 172.21.15.77:0/450962986 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556aa2007200 con 0x556aa22c1130 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a70ff9700 10 client.15429 handle_command_reply: tid=0 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a99e0ab80 1 client.15429 shutdown 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.236+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/450962986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 msgr2=0x556aa225ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa225ab10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3a84000c00 tx=0x7f3a84015e80 comp rx=0 tx=0).stop 2024-05-27T08:38:53.242 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/450962986 shutdown_connections 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x556aa22c1130 0x556aa22bfe40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556aa223c2d0 0x556aa2253c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556aa2236850 0x556aa225ab10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 --2- 172.21.15.77:0/450962986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x556aa2237210 0x556aa225b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/450962986 >> 172.21.15.77:0/450962986 conn(0x556aa21621f0 msgr2=0x556aa22360d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a70ff9700 0 client.15429 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/450962986 shutdown_connections 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 1 -- 172.21.15.77:0/450962986 wait complete. 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a99e0ab80 20 client.15429 trim_cache size 0 max 16384 2024-05-27T08:38:53.243 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "cluster_fsid": "c4eeaaa6-1c02-11ef-bc9b-c7b262605968", 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "whoami": 0, 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "id": 24434, 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "want_state": "up:active", 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "state": "up:active", 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "fs_name": "cephfs", 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "rank_uptime": 224.19563739099999, 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "mdsmap_epoch": 29, 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch": 86, 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_epoch_barrier": 85, 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout: "uptime": 239.58261918700001 2024-05-27T08:38:53.244 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:38:53.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 -- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3a740b2a80 msgr2=0x7f3a740b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3a740b2a80 0x7f3a740b4f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009a10 tx=0x7f3a84009190 comp rx=0 tx=0).stop 2024-05-27T08:38:53.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 -- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a881095b0 msgr2=0x7f3a88102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:38:53.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a881095b0 0x7f3a88102fc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b970 tx=0x7f3a8000bc80 comp rx=0 tx=0).stop 2024-05-27T08:38:53.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 -- 172.21.15.77:0/400862963 shutdown_connections 2024-05-27T08:38:53.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3a740b2a80 0x7f3a740b4f40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.245 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3a881095b0 0x7f3a88102fc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3a8806bc90 0x7f3a88102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 --2- 172.21.15.77:0/400862963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3a8806c650 0x7f3a88102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:38:53.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.237+0000 7f3a8f574700 1 -- 172.21.15.77:0/400862963 >> 172.21.15.77:0/400862963 conn(0x7f3a88079410 msgr2=0x7f3a881013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:38:53.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.238+0000 7f3a8f574700 1 -- 172.21.15.77:0/400862963 shutdown_connections 2024-05-27T08:38:53.246 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:38:53.238+0000 7f3a8f574700 1 -- 172.21.15.77:0/400862963 wait complete. 2024-05-27T08:38:53.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:53 smithi077 ceph-mon[32007]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-27T08:38:53.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:53 smithi092 ceph-mon[36049]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-27T08:38:53.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:53 smithi179 ceph-mon[36481]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-05-27T08:38:53.879 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-27T08:38:55.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:55 smithi077 ceph-mon[32007]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-27T08:38:55.736 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:55 smithi092 ceph-mon[36049]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-27T08:38:55.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:55 smithi179 ceph-mon[36481]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-05-27T08:38:57.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:57 smithi179 ceph-mon[36481]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:57.881 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:57 smithi077 ceph-mon[32007]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:57.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:57 smithi092 ceph-mon[36049]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:38:59.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:38:59 smithi179 ceph-mon[36481]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:38:59.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:38:59 smithi077 ceph-mon[32007]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:38:59.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:38:59 smithi092 ceph-mon[36049]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:39:01.129 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:39:00 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:39:00.714+0000 7f1ce94fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:39:01.130 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:39:00 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:39:00.714+0000 7f1ce94fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:39:01.130 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:39:00 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:39:00.714+0000 7f1ce94fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:39:01.130 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:39:00 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:39:00.714+0000 7f1ce94fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:39:01.130 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:39:00 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x[32237]: 2024-05-27T08:39:00.714+0000 7f1ce94fe700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-27T08:39:01.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:01 smithi179 ceph-mon[36481]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:01.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:01 smithi077 ceph-mon[32007]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:01.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:01 smithi092 ceph-mon[36049]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:03.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:03 smithi179 ceph-mon[36481]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-05-27T08:39:03.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:03 smithi179 ceph-mon[36481]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-05-27T08:39:03.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:03 smithi077 ceph-mon[32007]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-05-27T08:39:03.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:03 smithi077 ceph-mon[32007]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-05-27T08:39:03.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:03 smithi092 ceph-mon[36049]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 25 op/s 2024-05-27T08:39:03.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:03 smithi092 ceph-mon[36049]: mgrmap e25: x(active, since 46s), standbys: z, y 2024-05-27T08:39:05.857 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:05 smithi179 ceph-mon[36481]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-27T08:39:05.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:05 smithi077 ceph-mon[32007]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-27T08:39:05.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:05 smithi092 ceph-mon[36049]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-27T08:39:07.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:07 smithi179 ceph-mon[36481]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T08:39:07.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:07 smithi077 ceph-mon[32007]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T08:39:07.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:07 smithi092 ceph-mon[36049]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T08:39:08.880 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.4 2024-05-27T08:39:08.880 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:39:09.136 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:39:09.541 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:09 smithi077 ceph-mon[32007]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T08:39:09.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:09 smithi179 ceph-mon[36481]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T08:39:09.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:09 smithi092 ceph-mon[36049]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T08:39:10.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 -- 172.21.15.77:0/2670105759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6381095b0 msgr2=0x7fe638109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:10.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 --2- 172.21.15.77:0/2670105759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6381095b0 0x7fe638109990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe62800b0a0 tx=0x7fe62800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:39:10.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 -- 172.21.15.77:0/2670105759 shutdown_connections 2024-05-27T08:39:10.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 --2- 172.21.15.77:0/2670105759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe63806c650 0x7fe6381103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:10.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 --2- 172.21.15.77:0/2670105759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe6381095b0 0x7fe638109990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:10.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 --2- 172.21.15.77:0/2670105759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe63806bc90 0x7fe63806c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:10.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 -- 172.21.15.77:0/2670105759 >> 172.21.15.77:0/2670105759 conn(0x7fe638079410 msgr2=0x7fe638079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:10.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 -- 172.21.15.77:0/2670105759 shutdown_connections 2024-05-27T08:39:10.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 -- 172.21.15.77:0/2670105759 wait complete. 2024-05-27T08:39:10.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.738+0000 7fe63e141700 1 Processor -- start 2024-05-27T08:39:10.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe63e141700 1 -- start start 2024-05-27T08:39:10.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe63e141700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe63806bc90 0x7fe63819bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:10.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe63e141700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63806c650 0x7fe63819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:10.741 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe63e141700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6381095b0 0x7fe6381a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe63e141700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe6381136c0 con 0x7fe63806c650 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe63e141700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe638113540 con 0x7fe6381095b0 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe63e141700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe638113840 con 0x7fe63806bc90 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe636ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63806c650 0x7fe63819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe6377fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe63806bc90 0x7fe63819bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe636ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63806c650 0x7fe63819c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45144/0 (socket says 172.21.15.77:45144) 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe636ffd700 1 -- 172.21.15.77:0/2125856148 learned_addr learned my addr 172.21.15.77:0/2125856148 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe637fff700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6381095b0 0x7fe6381a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe636ffd700 1 -- 172.21.15.77:0/2125856148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe63806bc90 msgr2=0x7fe63819bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe636ffd700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe63806bc90 0x7fe63819bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe636ffd700 1 -- 172.21.15.77:0/2125856148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6381095b0 msgr2=0x7fe6381a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.739+0000 7fe636ffd700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6381095b0 0x7fe6381a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:10.742 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.740+0000 7fe636ffd700 1 -- 172.21.15.77:0/2125856148 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6381a0e00 con 0x7fe63806c650 2024-05-27T08:39:10.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.740+0000 7fe636ffd700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63806c650 0x7fe63819c330 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00c920 tx=0x7fe62c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:10.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.740+0000 7fe634ff9700 1 -- 172.21.15.77:0/2125856148 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c007b00 con 0x7fe63806c650 2024-05-27T08:39:10.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.740+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6381a10f0 con 0x7fe63806c650 2024-05-27T08:39:10.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.740+0000 7fe634ff9700 1 -- 172.21.15.77:0/2125856148 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62c004ce0 con 0x7fe63806c650 2024-05-27T08:39:10.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.740+0000 7fe634ff9700 1 -- 172.21.15.77:0/2125856148 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c005550 con 0x7fe63806c650 2024-05-27T08:39:10.743 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.740+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6381a8970 con 0x7fe63806c650 2024-05-27T08:39:10.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.741+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63806a230 con 0x7fe63806c650 2024-05-27T08:39:10.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.742+0000 7fe634ff9700 1 -- 172.21.15.77:0/2125856148 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fe62c007ca0 con 0x7fe63806c650 2024-05-27T08:39:10.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.743+0000 7fe634ff9700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe6200b2b10 0x7fe6200b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:10.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.743+0000 7fe6377fe700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe6200b2b10 0x7fe6200b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:10.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.743+0000 7fe634ff9700 1 -- 172.21.15.77:0/2125856148 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe62c0e29a0 con 0x7fe63806c650 2024-05-27T08:39:10.744 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.743+0000 7fe6377fe700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe6200b2b10 0x7fe6200b4fd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe628009e60 tx=0x7fe62800b7b0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:10.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:10.746+0000 7fe634ff9700 1 -- 172.21.15.77:0/2125856148 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe62c014030 con 0x7fe63806c650 2024-05-27T08:39:11.003 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.002+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe63819d150 con 0x7fe63806c650 2024-05-27T08:39:11.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.003+0000 7fe634ff9700 1 -- 172.21.15.77:0/2125856148 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7fe63819d150 con 0x7fe63806c650 2024-05-27T08:39:11.004 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:39:11.004 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:38:13.957300+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe6200b2b10 msgr2=0x7fe6200b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe6200b2b10 0x7fe6200b4fd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe628009e60 tx=0x7fe62800b7b0 comp rx=0 tx=0).stop 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63806c650 msgr2=0x7fe63819c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63806c650 0x7fe63819c330 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00c920 tx=0x7fe62c00cc30 comp rx=0 tx=0).stop 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 shutdown_connections 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe6200b2b10 0x7fe6200b4fd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe6381095b0 0x7fe6381a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe63806bc90 0x7fe63819bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 --2- 172.21.15.77:0/2125856148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe63806c650 0x7fe63819c330 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 >> 172.21.15.77:0/2125856148 conn(0x7fe638079410 msgr2=0x7fe63810e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 shutdown_connections 2024-05-27T08:39:11.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:11.005+0000 7fe63e141700 1 -- 172.21.15.77:0/2125856148 wait complete. 2024-05-27T08:39:11.008 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-05-27T08:39:11.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:11 smithi077 ceph-mon[32007]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T08:39:11.630 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:11 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2125856148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:39:11.788 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 90 --expiration 149.4 --await-for 45.0' 2024-05-27T08:39:11.788 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db / --timeout 90 --expiration 149.4 --await-for 45.0 2024-05-27T08:39:11.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:11 smithi179 ceph-mon[36481]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T08:39:11.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:11 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2125856148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:39:11.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:11 smithi092 ceph-mon[36049]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-05-27T08:39:11.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:11 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2125856148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:39:12.034 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:39:13.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 -- 172.21.15.77:0/2980555735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 msgr2=0x7f9678116620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 --2- 172.21.15.77:0/2980555735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 0x7f9678116620 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f967000b0a0 tx=0x7f967000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 -- 172.21.15.77:0/2980555735 shutdown_connections 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 --2- 172.21.15.77:0/2980555735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 0x7f9678116620 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 --2- 172.21.15.77:0/2980555735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9678077560 0x7f9678075770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 --2- 172.21.15.77:0/2980555735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9678075cb0 0x7f9678076130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 -- 172.21.15.77:0/2980555735 >> 172.21.15.77:0/2980555735 conn(0x7f967807a610 msgr2=0x7f967807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 -- 172.21.15.77:0/2980555735 shutdown_connections 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 -- 172.21.15.77:0/2980555735 wait complete. 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 Processor -- start 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.511+0000 7f96801da700 1 -- start start 2024-05-27T08:39:13.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f96801da700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9678075cb0 0x7f96781a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f96801da700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9678077560 0x7f96781a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f96801da700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 0x7f96781a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f96801da700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9678119d90 con 0x7f9678114230 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f96801da700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9678119c10 con 0x7f9678075cb0 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f96801da700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9678119f10 con 0x7f9678077560 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 0x7f96781a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 0x7f96781a49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45170/0 (socket says 172.21.15.77:45170) 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 -- 172.21.15.77:0/2476629606 learned_addr learned my addr 172.21.15.77:0/2476629606 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967df76700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9678075cb0 0x7f96781a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967d775700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9678077560 0x7f96781a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 -- 172.21.15.77:0/2476629606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9678077560 msgr2=0x7f96781a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9678077560 0x7f96781a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 -- 172.21.15.77:0/2476629606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9678075cb0 msgr2=0x7f96781a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9678075cb0 0x7f96781a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 -- 172.21.15.77:0/2476629606 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96781a5110 con 0x7f9678114230 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f967e777700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 0x7f96781a49d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9670006070 tx=0x7f9670004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f966effd700 1 -- 172.21.15.77:0/2476629606 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670021070 con 0x7f9678114230 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f96801da700 1 -- 172.21.15.77:0/2476629606 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96781a53a0 con 0x7f9678114230 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f966effd700 1 -- 172.21.15.77:0/2476629606 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9670007c80 con 0x7f9678114230 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f966effd700 1 -- 172.21.15.77:0/2476629606 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670004850 con 0x7f9678114230 2024-05-27T08:39:13.517 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.512+0000 7f96801da700 1 -- 172.21.15.77:0/2476629606 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96781acc80 con 0x7f9678114230 2024-05-27T08:39:13.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.515+0000 7f966effd700 1 -- 172.21.15.77:0/2476629606 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f9670003a00 con 0x7f9678114230 2024-05-27T08:39:13.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.515+0000 7f966effd700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f96640b2c40 0x7f96640b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f967df76700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f96640b2c40 0x7f96640b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f966effd700 1 -- 172.21.15.77:0/2476629606 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f967000f020 con 0x7f9678114230 2024-05-27T08:39:13.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f968aa70b80 1 Processor -- start 2024-05-27T08:39:13.520 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f967df76700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f96640b2c40 0x7f96640b5100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f967400ac50 tx=0x7f967400a380 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f968aa70b80 1 -- start start 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087953720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x56008794ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f968aa70b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5600878ace20 con 0x56008794e330 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f968aa70b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56008789c9c0 con 0x560087953340 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.516+0000 7f968aa70b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5600878a0a00 con 0x56008794d970 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45176/0 (socket says 172.21.15.77:45176) 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 -- 172.21.15.77:0/1130286573 learned_addr learned my addr 172.21.15.77:0/1130286573 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967df76700 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087953720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 -- 172.21.15.77:0/1130286573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 msgr2=0x56008794ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:13.521 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967d775700 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x56008794ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x56008794ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 -- 172.21.15.77:0/1130286573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 msgr2=0x560087953720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087953720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 -- 172.21.15.77:0/1130286573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600877eca60 con 0x56008794e330 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967df76700 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087953720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f967e777700 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795a170 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9670009fd0 tx=0x7f967000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75914d24567bbae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f9663fff700 1 -- 172.21.15.77:0/1130286573 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700af470 con 0x56008794e330 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f9663fff700 1 -- 172.21.15.77:0/1130286573 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96700b3e90 con 0x56008794e330 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.517+0000 7f9663fff700 1 -- 172.21.15.77:0/1130286573 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700ab580 con 0x56008794e330 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 -- 172.21.15.77:0/1130286573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 msgr2=0x56008795a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795a170 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9670009fd0 tx=0x7f967000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 -- 172.21.15.77:0/1130286573 shutdown_connections 2024-05-27T08:39:13.522 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795a170 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087953720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1130286573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x56008794ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 -- 172.21.15.77:0/1130286573 >> 172.21.15.77:0/1130286573 conn(0x56008787b210 msgr2=0x560087948280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:13.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 -- 172.21.15.77:0/1130286573 shutdown_connections 2024-05-27T08:39:13.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 -- 172.21.15.77:0/1130286573 wait complete. 2024-05-27T08:39:13.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 -- start start 2024-05-27T08:39:13.523 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.518+0000 7f968aa70b80 1 Processor -- start 2024-05-27T08:39:13.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x560087959f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x560087957230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087957770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.526 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f968aa70b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5600877a3130 con 0x56008794e330 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f968aa70b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56008783e950 con 0x560087953340 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f968aa70b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560087953ee0 con 0x56008794d970 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f968aa70b80 10 client.0 _ll_register_callbacks cb 0x560087837ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967d775700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x560087957230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967d775700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x560087957230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45188/0 (socket says 172.21.15.77:45188) 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967d775700 1 -- 172.21.15.77:0/1879365361 learned_addr learned my addr 172.21.15.77:0/1879365361 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967df76700 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x560087959f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967e777700 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087957770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967d775700 1 -- 172.21.15.77:0/1879365361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 msgr2=0x560087959f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967d775700 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x560087959f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967d775700 1 -- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 msgr2=0x560087957770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967d775700 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087957770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.527 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967d775700 1 -- 172.21.15.77:0/1879365361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008783b820 con 0x56008794e330 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.519+0000 7f967e777700 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087957770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f967d775700 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x560087957230 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f966800e1f0 tx=0x7f966800e500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400f9a38006e847f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f96617fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f96617fa700 1 -- 172.21.15.77:0/1879365361 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9668017020 con 0x56008794e330 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f96617fa700 1 -- 172.21.15.77:0/1879365361 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f966800ee30 con 0x56008794e330 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 1 -- 172.21.15.77:0/1879365361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96700b3e90 con 0x56008794e330 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f96617fa700 1 -- 172.21.15.77:0/1879365361 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9668022050 con 0x56008794e330 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 10 client.15453 fetch_fsmap learned FSMap version 29 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 1 -- 172.21.15.77:0/1879365361 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560087964c80 con 0x56008794e330 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f96617fa700 1 -- 172.21.15.77:0/1879365361 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96700b3e90 con 0x56008794e330 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f96617fa700 1 -- 172.21.15.77:0/1879365361 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9668012050 con 0x56008794e330 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 10 client.15453 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:39:13.528 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 10 client.15453 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 20 client.15453 populate_metadata read hostname 'smithi077' 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x560087967360 0x560087969750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 4 client.15453 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.520+0000 7f968aa70b80 1 -- 172.21.15.77:0/1879365361 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560087919b70 con 0x560087967360 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.521+0000 7f967df76700 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x560087967360 0x560087969750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.521+0000 7f96617fa700 10 client.15453 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.521+0000 7f967df76700 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x560087967360 0x560087969750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7186668917537b4b server_cookie=231f808bc5e33226 in_seq=0 out_seq=0 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f96617fa700 1 -- 172.21.15.77:0/1879365361 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560087919b70 con 0x560087967360 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f96617fa700 10 client.15453 handle_command_reply: tid=0 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 client.15453 shutdown 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 -- 172.21.15.77:0/1879365361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 msgr2=0x560087957230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x560087957230 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f966800e1f0 tx=0x7f966800e500 comp rx=0 tx=0).stop 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 -- 172.21.15.77:0/1879365361 shutdown_connections 2024-05-27T08:39:13.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x560087967360 0x560087969750 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087957770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x560087959f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f96617fa700 0 client.15453 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:39:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1879365361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x560087957230 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 -- 172.21.15.77:0/1879365361 >> 172.21.15.77:0/1879365361 conn(0x56008787b210 msgr2=0x560087942d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 -- 172.21.15.77:0/1879365361 shutdown_connections 2024-05-27T08:39:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 1 -- 172.21.15.77:0/1879365361 wait complete. 2024-05-27T08:39:13.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.522+0000 7f968aa70b80 20 client.15453 trim_cache size 0 max 16384 2024-05-27T08:39:13.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f968aa70b80 1 Processor -- start 2024-05-27T08:39:13.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f968aa70b80 1 -- start start 2024-05-27T08:39:13.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794d970 0x56008794d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.545 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794e330 0x560087963610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087959a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f968aa70b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b4240 con 0x56008794d970 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f968aa70b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b5d80 con 0x560087953340 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f968aa70b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b4910 con 0x56008794e330 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.542+0000 7f967df76700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794d970 0x56008794d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967df76700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794d970 0x56008794d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45190/0 (socket says 172.21.15.77:45190) 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967df76700 1 -- 172.21.15.77:0/1737166937 learned_addr learned my addr 172.21.15.77:0/1737166937 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967d775700 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794e330 0x560087963610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967e777700 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087959a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967df76700 1 -- 172.21.15.77:0/1737166937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794e330 msgr2=0x560087963610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967df76700 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794e330 0x560087963610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967df76700 1 -- 172.21.15.77:0/1737166937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 msgr2=0x560087959a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:13.546 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967df76700 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087959a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967df76700 1 -- 172.21.15.77:0/1737166937 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600879557a0 con 0x56008794d970 2024-05-27T08:39:13.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967e777700 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087959a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:39:13.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f967df76700 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794d970 0x56008794d130 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f967400aa30 tx=0x7f9674012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67362bca8e3b7ea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:13.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f96637fe700 1 -- 172.21.15.77:0/1737166937 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967402b070 con 0x56008794d970 2024-05-27T08:39:13.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f96637fe700 1 -- 172.21.15.77:0/1737166937 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9674014070 con 0x56008794d970 2024-05-27T08:39:13.547 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.543+0000 7f96637fe700 1 -- 172.21.15.77:0/1737166937 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674016070 con 0x56008794d970 2024-05-27T08:39:13.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 -- 172.21.15.77:0/1737166937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794d970 msgr2=0x56008794d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794d970 0x56008794d130 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f967400aa30 tx=0x7f9674012000 comp rx=0 tx=0).stop 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 -- 172.21.15.77:0/1737166937 shutdown_connections 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x560087959a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794d970 0x56008794d130 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1737166937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794e330 0x560087963610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 -- 172.21.15.77:0/1737166937 >> 172.21.15.77:0/1737166937 conn(0x56008787b210 msgr2=0x560087950e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 -- 172.21.15.77:0/1737166937 shutdown_connections 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 -- 172.21.15.77:0/1737166937 wait complete. 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 -- start start 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.544+0000 7f968aa70b80 1 Processor -- start 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x56008795c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.550 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f968aa70b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x5600879d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f968aa70b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b56b0 con 0x56008794e330 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f968aa70b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b4fe0 con 0x560087953340 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f968aa70b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f96781b2200 con 0x56008794d970 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f968aa70b80 10 client.0 _ll_register_callbacks cb 0x560087837ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967d775700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967d775700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:45196/0 (socket says 172.21.15.77:45196) 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967d775700 1 -- 172.21.15.77:0/1121731836 learned_addr learned my addr 172.21.15.77:0/1121731836 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967df76700 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x56008795c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967e777700 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x5600879d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967d775700 1 -- 172.21.15.77:0/1121731836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 msgr2=0x56008795c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967d775700 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x56008795c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967d775700 1 -- 172.21.15.77:0/1121731836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 msgr2=0x5600879d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967d775700 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x5600879d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.545+0000 7f967d775700 1 -- 172.21.15.77:0/1121731836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600879d24f0 con 0x56008794e330 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.546+0000 7f967d775700 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795cdf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x560087940990 tx=0x7f9668004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634a876bd6e212 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.546+0000 7f96617fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f96617fa700 1 -- 172.21.15.77:0/1121731836 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966802c070 con 0x56008794e330 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f96617fa700 1 -- 172.21.15.77:0/1121731836 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f966801b8e0 con 0x56008794e330 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 1 -- 172.21.15.77:0/1121731836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9674014070 con 0x56008794e330 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f96617fa700 1 -- 172.21.15.77:0/1121731836 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9668017040 con 0x56008794e330 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 10 client.15459 fetch_fsmap learned FSMap version 29 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 1 -- 172.21.15.77:0/1121731836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5600879d3640 con 0x56008794e330 2024-05-27T08:39:13.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f96617fa700 1 -- 172.21.15.77:0/1121731836 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9674014070 con 0x56008794e330 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f96617fa700 1 -- 172.21.15.77:0/1121731836 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9668012050 con 0x56008794e330 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 10 client.15459 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 20 client.15459 populate_metadata read hostname 'smithi077' 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5600879d3d90 0x5600879d6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 4 client.15459 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.4, "await_for": 45.0}] 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.547+0000 7f968aa70b80 1 -- 172.21.15.77:0/1121731836 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.4, "await_for": 45.0}) -- 0x5600878aed70 con 0x5600879d3d90 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.548+0000 7f967df76700 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5600879d3d90 0x5600879d6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:13.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.548+0000 7f967df76700 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5600879d3d90 0x5600879d6180 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f323ae318a98d0b server_cookie=1a7e978cf0255296 in_seq=0 out_seq=0 2024-05-27T08:39:13.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.548+0000 7f96617fa700 10 client.15459 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96617fa700 1 -- 172.21.15.77:0/1121731836 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5600878aed70 con 0x5600879d3d90 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96617fa700 10 client.15459 handle_command_reply: tid=0 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 client.15459 shutdown 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 -- 172.21.15.77:0/1121731836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 msgr2=0x56008795cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795cdf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x560087940990 tx=0x7f9668004750 comp rx=0 tx=0).stop 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 -- 172.21.15.77:0/1121731836 shutdown_connections 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5600879d3d90 0x5600879d6180 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x560087953340 0x5600879d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56008794d970 0x56008795c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 --2- 172.21.15.77:0/1121731836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56008794e330 0x56008795cdf0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96617fa700 0 client.15459 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 -- 172.21.15.77:0/1121731836 >> 172.21.15.77:0/1121731836 conn(0x56008787b210 msgr2=0x5600879516b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 -- 172.21.15.77:0/1121731836 shutdown_connections 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 1 -- 172.21.15.77:0/1121731836 wait complete. 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f968aa70b80 20 client.15459 trim_cache size 0 max 16384 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 29, 2024-05-27T08:39:13.565 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:39:13.566 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 2, 2024-05-27T08:39:13.566 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:39:13.566 INFO:teuthology.orchestra.run.smithi077.stdout: "595a5120": { 2024-05-27T08:39:13.566 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 2, 2024-05-27T08:39:13.566 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:39:13.566 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:39:13.566 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 149.4, 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:13.567 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:13.568 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:13.568 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:13.568 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:39:13.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 -- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f96640b2c40 msgr2=0x7f96640b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f96640b2c40 0x7f96640b5100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f967400ac50 tx=0x7f967400a380 comp rx=0 tx=0).stop 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 -- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 msgr2=0x7f96781a49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 0x7f96781a49d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9670006070 tx=0x7f9670004e60 comp rx=0 tx=0).stop 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 -- 172.21.15.77:0/2476629606 shutdown_connections 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f96640b2c40 0x7f96640b5100 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9678114230 0x7f96781a49d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9678075cb0 0x7f96781a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 --2- 172.21.15.77:0/2476629606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9678077560 0x7f96781a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 -- 172.21.15.77:0/2476629606 >> 172.21.15.77:0/2476629606 conn(0x7f967807a610 msgr2=0x7f9678112df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:13.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 -- 172.21.15.77:0/2476629606 shutdown_connections 2024-05-27T08:39:13.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:13.562+0000 7f96801da700 1 -- 172.21.15.77:0/2476629606 wait complete. 2024-05-27T08:39:13.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:13 smithi179 ceph-mon[36481]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T08:39:13.870 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:13 smithi077 ceph-mon[32007]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T08:39:13.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:13 smithi092 ceph-mon[36049]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-05-27T08:39:14.385 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '595a5120', quiesce took 5.5 seconds. Will release after: 23.9 2024-05-27T08:39:15.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:15 smithi179 ceph-mon[36481]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-27T08:39:15.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:15 smithi077 ceph-mon[32007]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-27T08:39:15.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:15 smithi092 ceph-mon[36049]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-05-27T08:39:17.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:17 smithi179 ceph-mon[36481]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:39:17.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:17 smithi077 ceph-mon[32007]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:39:17.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:17 smithi092 ceph-mon[36049]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 8 op/s 2024-05-27T08:39:19.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:19 smithi179 ceph-mon[36481]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-27T08:39:19.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:19 smithi077 ceph-mon[32007]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-27T08:39:19.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:19 smithi092 ceph-mon[36049]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-05-27T08:39:21.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:21 smithi179 ceph-mon[36481]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:21 smithi077 ceph-mon[32007]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:21.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:21 smithi092 ceph-mon[36049]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:22.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:22 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:39:22.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:22 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:22.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:22 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:22.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:22 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:39:22.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:22 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:22.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:22 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:22.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:22 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:39:22.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:22 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:22.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:22 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:23.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:23 smithi179 ceph-mon[36481]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-27T08:39:23.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:23 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:39:23.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:23 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:39:23.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:23 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:23.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:23 smithi077 ceph-mon[32007]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-27T08:39:23.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:23 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:39:23.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:23 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:39:23.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:23 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:23.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:23 smithi092 ceph-mon[36049]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 938 B/s wr, 12 op/s 2024-05-27T08:39:23.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:23 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:39:23.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:23 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:39:23.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:23 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:39:25.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:25 smithi179 ceph-mon[36481]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:25.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:25 smithi077 ceph-mon[32007]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:25.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:25 smithi092 ceph-mon[36049]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:27.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:27 smithi179 ceph-mon[36481]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:27.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:27 smithi077 ceph-mon[32007]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:27 smithi092 ceph-mon[36049]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 853 B/s wr, 8 op/s 2024-05-27T08:39:29.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:29 smithi179 ceph-mon[36481]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-27T08:39:29.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:29 smithi077 ceph-mon[32007]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-27T08:39:29.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:29 smithi092 ceph-mon[36049]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-05-27T08:39:31.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:31 smithi179 ceph-mon[36481]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-27T08:39:31.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:31 smithi077 ceph-mon[32007]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-27T08:39:31.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:31 smithi092 ceph-mon[36049]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-27T08:39:33.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:33 smithi179 ceph-mon[36481]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-27T08:39:33.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:33 smithi077 ceph-mon[32007]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-27T08:39:33.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:33 smithi092 ceph-mon[36049]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-05-27T08:39:35.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:35 smithi179 ceph-mon[36481]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:35.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:35 smithi077 ceph-mon[32007]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:35.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:35 smithi092 ceph-mon[36049]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:37.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:37 smithi179 ceph-mon[36481]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:37.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:37 smithi077 ceph-mon[32007]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:37.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:37 smithi092 ceph-mon[36049]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:38.286 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:39:38.605 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:39:39.847 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:39 smithi077 ceph-mon[32007]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:39:39.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:39 smithi179 ceph-mon[36481]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:39:39.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:39 smithi092 ceph-mon[36049]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:39:40.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.406+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2049641800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 msgr2=0x7f59cc1122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:40.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.406+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2049641800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 0x7f59cc1122c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b0a0 tx=0x7f59c400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.406+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2049641800 shutdown_connections 2024-05-27T08:39:40.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.406+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2049641800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 0x7f59cc1122c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.406+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2049641800 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59cc104450 0x7f59cc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.406+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2049641800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc101b30 0x7f59cc101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.406+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2049641800 >> 172.21.15.77:0/2049641800 conn(0x7f59cc078ab0 msgr2=0x7f59cc0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:40.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.407+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2049641800 shutdown_connections 2024-05-27T08:39:40.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.407+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2049641800 wait complete. 2024-05-27T08:39:40.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.407+0000 7f59d2ac8700 1 Processor -- start 2024-05-27T08:39:40.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.407+0000 7f59d2ac8700 1 -- start start 2024-05-27T08:39:40.409 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d2ac8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc101b30 0x7f59cc19bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d2ac8700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 0x7f59cc19c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d2ac8700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59cc104450 0x7f59cc1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59cbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 0x7f59cc19c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d0864700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc101b30 0x7f59cc19bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59cbfff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 0x7f59cc19c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52826/0 (socket says 172.21.15.77:52826) 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59cbfff700 1 -- 172.21.15.77:0/2876185037 learned_addr learned my addr 172.21.15.77:0/2876185037 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc115900 con 0x7f59cc1025d0 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc115780 con 0x7f59cc104450 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc115a80 con 0x7f59cc101b30 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d1065700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59cc104450 0x7f59cc1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d1065700 1 -- 172.21.15.77:0/2876185037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc101b30 msgr2=0x7f59cc19bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d1065700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc101b30 0x7f59cc19bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.410 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d1065700 1 -- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 msgr2=0x7f59cc19c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:40.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d1065700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 0x7f59cc19c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d1065700 1 -- 172.21.15.77:0/2876185037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc1a0da0 con 0x7f59cc104450 2024-05-27T08:39:40.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.408+0000 7f59d0864700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc101b30 0x7f59cc19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:39:40.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.409+0000 7f59d1065700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59cc104450 0x7f59cc1a0660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b070 tx=0x7f59c4004ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:40.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.410+0000 7f59c9ffb700 1 -- 172.21.15.77:0/2876185037 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4021070 con 0x7f59cc104450 2024-05-27T08:39:40.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.410+0000 7f59c9ffb700 1 -- 172.21.15.77:0/2876185037 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c4007b80 con 0x7f59cc104450 2024-05-27T08:39:40.411 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.410+0000 7f59c9ffb700 1 -- 172.21.15.77:0/2876185037 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c401de60 con 0x7f59cc104450 2024-05-27T08:39:40.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.410+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc1a1030 con 0x7f59cc104450 2024-05-27T08:39:40.412 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.411+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc1a8910 con 0x7f59cc104450 2024-05-27T08:39:40.413 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.412+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc102b40 con 0x7f59cc104450 2024-05-27T08:39:40.415 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.414+0000 7f59c9ffb700 1 -- 172.21.15.77:0/2876185037 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f59c4024070 con 0x7f59cc104450 2024-05-27T08:39:40.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.415+0000 7f59c9ffb700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f59bc0b2b90 0x7f59bc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:40.416 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.416+0000 7f59c9ffb700 1 -- 172.21.15.77:0/2876185037 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f59c400f020 con 0x7f59cc104450 2024-05-27T08:39:40.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.416+0000 7f59d0864700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f59bc0b2b90 0x7f59bc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:40.417 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.416+0000 7f59d0864700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f59bc0b2b90 0x7f59bc0b5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009b50 tx=0x7f59c0009550 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:40.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.418+0000 7f59c9ffb700 1 -- 172.21.15.77:0/2876185037 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59c402b020 con 0x7f59cc104450 2024-05-27T08:39:40.749 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.748+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59cc02e560 con 0x7f59cc104450 2024-05-27T08:39:40.750 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.749+0000 7f59c9ffb700 1 -- 172.21.15.77:0/2876185037 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11927 (secure 0 0 0) 0x7f59c40ab9a0 con 0x7f59cc104450 2024-05-27T08:39:40.750 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:39:40.751 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:38:13.957300+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f59bc0b2b90 msgr2=0x7f59bc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f59bc0b2b90 0x7f59bc0b5050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009b50 tx=0x7f59c0009550 comp rx=0 tx=0).stop 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59cc104450 msgr2=0x7f59cc1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59cc104450 0x7f59cc1a0660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59c400b070 tx=0x7f59c4004ce0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 shutdown_connections 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f59bc0b2b90 0x7f59bc0b5050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f59cc104450 0x7f59cc1a0660 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59cc101b30 0x7f59cc19bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 --2- 172.21.15.77:0/2876185037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59cc1025d0 0x7f59cc19c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 >> 172.21.15.77:0/2876185037 conn(0x7f59cc078ab0 msgr2=0x7f59cc0ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 shutdown_connections 2024-05-27T08:39:40.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:40.752+0000 7f59d2ac8700 1 -- 172.21.15.77:0/2876185037 wait complete. 2024-05-27T08:39:40.754 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 29 2024-05-27T08:39:41.383 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 595a5120 --release --await' 2024-05-27T08:39:41.384 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db --set-id 595a5120 --release --await 2024-05-27T08:39:41.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:41 smithi077 ceph-mon[32007]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:41.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:41 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2876185037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:39:41.746 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:39:41.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:41 smithi179 ceph-mon[36481]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:41.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:41 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2876185037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:39:41.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:41 smithi092 ceph-mon[36049]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:41.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:41 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2876185037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:39:43.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 -- 172.21.15.77:0/2823416737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00104200 msgr2=0x7f4f001081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 --2- 172.21.15.77:0/2823416737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00104200 0x7f4f001081f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00099c0 tx=0x7f4ef0009cd0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 -- 172.21.15.77:0/2823416737 shutdown_connections 2024-05-27T08:39:43.548 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 --2- 172.21.15.77:0/2823416737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f00108920 0x7f4f001104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 --2- 172.21.15.77:0/2823416737 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f00103850 0x7f4f00103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 --2- 172.21.15.77:0/2823416737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00104200 0x7f4f001081f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 -- 172.21.15.77:0/2823416737 >> 172.21.15.77:0/2823416737 conn(0x7f4f0007a5d0 msgr2=0x7f4f0007a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 -- 172.21.15.77:0/2823416737 shutdown_connections 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.546+0000 7f4f073d6700 1 -- 172.21.15.77:0/2823416737 wait complete. 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f073d6700 1 Processor -- start 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f073d6700 1 -- start start 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f073d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f00103850 0x7f4f00076d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f073d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f00104200 0x7f4f000772a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f073d6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00108920 0x7f4f0006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f073d6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001136e0 con 0x7f4f00108920 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f073d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00113560 con 0x7f4f00104200 2024-05-27T08:39:43.549 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f073d6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00113860 con 0x7f4f00103850 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f05973700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00108920 0x7f4f0006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.547+0000 7f4f04971700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f00104200 0x7f4f000772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05172700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f00103850 0x7f4f00076d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05973700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00108920 0x7f4f0006ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52842/0 (socket says 172.21.15.77:52842) 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05172700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f00103850 0x7f4f00076d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.77:56336/0 (socket says 172.21.15.77:56336) 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05973700 1 -- 172.21.15.77:0/4262579867 learned_addr learned my addr 172.21.15.77:0/4262579867 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05973700 1 -- 172.21.15.77:0/4262579867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f00103850 msgr2=0x7f4f00076d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05973700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f00103850 0x7f4f00076d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.551 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05973700 1 -- 172.21.15.77:0/4262579867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f00104200 msgr2=0x7f4f000772a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05973700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f00104200 0x7f4f000772a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05973700 1 -- 172.21.15.77:0/4262579867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0006e6b0 con 0x7f4f00108920 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f04971700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f00104200 0x7f4f000772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f05973700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00108920 0x7f4f0006ddb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800ef30 tx=0x7f4ef800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4ef67fc700 1 -- 172.21.15.77:0/4262579867 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80092e0 con 0x7f4f00108920 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4f073d6700 1 -- 172.21.15.77:0/4262579867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0006e940 con 0x7f4f00108920 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4ef67fc700 1 -- 172.21.15.77:0/4262579867 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef8004bb0 con 0x7f4f00108920 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.548+0000 7f4ef67fc700 1 -- 172.21.15.77:0/4262579867 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8004020 con 0x7f4f00108920 2024-05-27T08:39:43.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.549+0000 7f4f073d6700 1 -- 172.21.15.77:0/4262579867 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f00072980 con 0x7f4f00108920 2024-05-27T08:39:43.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.552+0000 7f4ef67fc700 1 -- 172.21.15.77:0/4262579867 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f4ef801b030 con 0x7f4f00108920 2024-05-27T08:39:43.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.553+0000 7f4ef67fc700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f4eec0b2b10 0x7f4eec0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f05172700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f4eec0b2b10 0x7f4eec0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4ef67fc700 1 -- 172.21.15.77:0/4262579867 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4ef80e2300 con 0x7f4f00108920 2024-05-27T08:39:43.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f11c6cb80 1 Processor -- start 2024-05-27T08:39:43.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f05172700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f4eec0b2b10 0x7f4eec0b4fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005950 tx=0x7f4efc00b410 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:43.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f11c6cb80 1 -- start start 2024-05-27T08:39:43.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe89690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe838e0 0x557dfbe83d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557dfbde2d90 con 0x557dfbe838e0 2024-05-27T08:39:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557dfbdd2930 con 0x557dfbe842a0 2024-05-27T08:39:43.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.554+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557dfbdd6970 con 0x557dfbe892b0 2024-05-27T08:39:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f04971700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe838e0 0x557dfbe83d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f04971700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe838e0 0x557dfbe83d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52858/0 (socket says 172.21.15.77:52858) 2024-05-27T08:39:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f04971700 1 -- 172.21.15.77:0/4066996240 learned_addr learned my addr 172.21.15.77:0/4066996240 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f05172700 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe89690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f05973700 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f05172700 1 -- 172.21.15.77:0/4066996240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 msgr2=0x557dfbe900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f05172700 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.557 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f05172700 1 -- 172.21.15.77:0/4066996240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe838e0 msgr2=0x557dfbe83d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:43.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f05172700 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe838e0 0x557dfbe83d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.555+0000 7f4f05172700 1 -- 172.21.15.77:0/4066996240 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dfbd229d0 con 0x557dfbe892b0 2024-05-27T08:39:43.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.556+0000 7f4f05172700 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe89690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005d90 tx=0x7f4efc013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aa0ab690a29fcf8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:43.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.556+0000 7f4eeb7fe700 1 -- 172.21.15.77:0/4066996240 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc011b80 con 0x557dfbe892b0 2024-05-27T08:39:43.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.556+0000 7f4eeb7fe700 1 -- 172.21.15.77:0/4066996240 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4efc011d20 con 0x557dfbe892b0 2024-05-27T08:39:43.558 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.556+0000 7f4eeb7fe700 1 -- 172.21.15.77:0/4066996240 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc01e420 con 0x557dfbe892b0 2024-05-27T08:39:43.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/4066996240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 msgr2=0x557dfbe89690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe89690 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005d90 tx=0x7f4efc013000 comp rx=0 tx=0).stop 2024-05-27T08:39:43.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/4066996240 shutdown_connections 2024-05-27T08:39:43.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe89690 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/4066996240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe838e0 0x557dfbe83d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/4066996240 >> 172.21.15.77:0/4066996240 conn(0x557dfbdb1180 msgr2=0x557dfbe7e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/4066996240 shutdown_connections 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/4066996240 wait complete. 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 -- start start 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.557+0000 7f4f11c6cb80 1 Processor -- start 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe838e0 0x557dfbe8fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe8d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe8d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557dfbcd90a0 con 0x557dfbe842a0 2024-05-27T08:39:43.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x557dfbd748c0 con 0x557dfbe838e0 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557dfbe906e0 con 0x557dfbe892b0 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f11c6cb80 10 client.0 _ll_register_callbacks cb 0x557dfbd6da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f04971700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe8d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f05172700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe838e0 0x557dfbe8fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f05172700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe838e0 0x557dfbe8fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:56640/0 (socket says 172.21.15.77:56640) 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f04971700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe8d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52868/0 (socket says 172.21.15.77:52868) 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.558+0000 7f4f05172700 1 -- 172.21.15.77:0/432725538 learned_addr learned my addr 172.21.15.77:0/432725538 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4f05973700 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe8d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4f05172700 1 -- 172.21.15.77:0/432725538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 msgr2=0x557dfbe8d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4f05172700 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe8d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4f05172700 1 -- 172.21.15.77:0/432725538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 msgr2=0x557dfbe8d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4f05172700 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe8d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4f05172700 1 -- 172.21.15.77:0/432725538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dfbd71790 con 0x557dfbe838e0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4f05172700 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe838e0 0x557dfbe8fd90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005380 tx=0x7f4efc005610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d5d52407e1290fb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4ee8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/432725538 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc036060 con 0x557dfbe838e0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/432725538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4efc011d20 con 0x557dfbe838e0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.559+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/432725538 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4efc004b20 con 0x557dfbe838e0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/432725538 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4efc004d20 con 0x557dfbe838e0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f11c6cb80 10 client.25025 fetch_fsmap learned FSMap version 29 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/432725538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557dfbe9ae60 con 0x557dfbe838e0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/432725538 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4efc011d20 con 0x557dfbe838e0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/432725538 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4efc01e420 con 0x557dfbe838e0 2024-05-27T08:39:43.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f11c6cb80 10 client.25025 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:39:43.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f11c6cb80 10 client.25025 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:39:43.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f11c6cb80 20 client.25025 populate_metadata read hostname 'smithi077' 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x557dfbe9d390 0x557dfbe9f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f11c6cb80 4 client.25025 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/432725538 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dfbe4fae0 con 0x557dfbe9d390 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.560+0000 7f4f04971700 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x557dfbe9d390 0x557dfbe9f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.561+0000 7f4f04971700 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x557dfbe9d390 0x557dfbe9f780 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94a9011374a6b68 server_cookie=fd4cff579eefad66 in_seq=0 out_seq=0 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.561+0000 7f4ee8ff9700 10 client.25025 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/432725538 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557dfbe4fae0 con 0x557dfbe9d390 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4ee8ff9700 10 client.25025 handle_command_reply: tid=0 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 client.25025 shutdown 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/432725538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe838e0 msgr2=0x557dfbe8fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe838e0 0x557dfbe8fd90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005380 tx=0x7f4efc005610 comp rx=0 tx=0).stop 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/432725538 shutdown_connections 2024-05-27T08:39:43.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x557dfbe9d390 0x557dfbe9f780 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe892b0 0x557dfbe8d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe838e0 0x557dfbe8fd90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/432725538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe8d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/432725538 >> 172.21.15.77:0/432725538 conn(0x557dfbdb1180 msgr2=0x557dfbe78ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4ee8ff9700 0 client.25025 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:39:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/432725538 shutdown_connections 2024-05-27T08:39:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/432725538 wait complete. 2024-05-27T08:39:43.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.562+0000 7f4f11c6cb80 20 client.25025 trim_cache size 0 max 16384 2024-05-27T08:39:43.603 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.602+0000 7f4f11c6cb80 1 Processor -- start 2024-05-27T08:39:43.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.602+0000 7f4f11c6cb80 1 -- start start 2024-05-27T08:39:43.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 0x557dfbe832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe8f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe892b0 0x557dfbea4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001bcbc0 con 0x557dfbe892b0 2024-05-27T08:39:43.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001be700 con 0x557dfbe842a0 2024-05-27T08:39:43.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001bd290 con 0x557dfbe838e0 2024-05-27T08:39:43.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f05973700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe892b0 0x557dfbea4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f04971700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe8f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f04971700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe8f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:56642/0 (socket says 172.21.15.77:56642) 2024-05-27T08:39:43.605 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f04971700 1 -- 172.21.15.77:0/2161347534 learned_addr learned my addr 172.21.15.77:0/2161347534 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f05172700 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 0x557dfbe832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f04971700 1 -- 172.21.15.77:0/2161347534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 msgr2=0x557dfbe832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f04971700 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 0x557dfbe832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f04971700 1 -- 172.21.15.77:0/2161347534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe892b0 msgr2=0x557dfbea4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f04971700 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe892b0 0x557dfbea4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f04971700 1 -- 172.21.15.77:0/2161347534 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dfbe80390 con 0x557dfbe842a0 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.603+0000 7f4f05973700 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe892b0 0x557dfbea4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.604+0000 7f4f04971700 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe8f8d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009750 tx=0x7f4ef001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c13d10fc1b80a44 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.604+0000 7f4eeaffd700 1 -- 172.21.15.77:0/2161347534 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00050b0 con 0x557dfbe842a0 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.604+0000 7f4eeaffd700 1 -- 172.21.15.77:0/2161347534 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef002a070 con 0x557dfbe842a0 2024-05-27T08:39:43.606 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.604+0000 7f4eeaffd700 1 -- 172.21.15.77:0/2161347534 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000f460 con 0x557dfbe842a0 2024-05-27T08:39:43.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/2161347534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 msgr2=0x557dfbe8f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe8f8d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009750 tx=0x7f4ef001a040 comp rx=0 tx=0).stop 2024-05-27T08:39:43.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/2161347534 shutdown_connections 2024-05-27T08:39:43.607 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe892b0 0x557dfbea4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 0x557dfbe832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/2161347534 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe842a0 0x557dfbe8f8d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/2161347534 >> 172.21.15.77:0/2161347534 conn(0x557dfbdb1180 msgr2=0x557dfbe87620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/2161347534 shutdown_connections 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.605+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/2161347534 wait complete. 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f11c6cb80 1 -- start start 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f11c6cb80 1 Processor -- start 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 0x557dfbe95970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe95eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f11c6cb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe892b0 0x557dfbe92940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001be030 con 0x557dfbe842a0 2024-05-27T08:39:43.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001bd960 con 0x557dfbe892b0 2024-05-27T08:39:43.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f11c6cb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f001bab60 con 0x557dfbe838e0 2024-05-27T08:39:43.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f04971700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe95eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.606+0000 7f4f04971700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe95eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:52896/0 (socket says 172.21.15.77:52896) 2024-05-27T08:39:43.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f04971700 1 -- 172.21.15.77:0/3251877115 learned_addr learned my addr 172.21.15.77:0/3251877115 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:39:43.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f11c6cb80 10 client.0 _ll_register_callbacks cb 0x557dfbd6da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f05172700 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 0x557dfbe95970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f04971700 1 -- 172.21.15.77:0/3251877115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 msgr2=0x557dfbe95970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f04971700 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 0x557dfbe95970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f04971700 1 -- 172.21.15.77:0/3251877115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe892b0 msgr2=0x557dfbe92940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f04971700 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe892b0 0x557dfbe92940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f04971700 1 -- 172.21.15.77:0/3251877115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dfbe92e80 con 0x557dfbe842a0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f04971700 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe95eb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0003760 tx=0x7f4ef000b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6938f5a49064d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4ee8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/3251877115 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0005580 con 0x557dfbe842a0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/3251877115 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f4ef00037c0 con 0x557dfbe842a0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.607+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/3251877115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef002a070 con 0x557dfbe842a0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.608+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/3251877115 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ef0046050 con 0x557dfbe842a0 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.608+0000 7f4f11c6cb80 10 client.15483 fetch_fsmap learned FSMap version 29 2024-05-27T08:39:43.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.608+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/3251877115 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557dfbf08580 con 0x557dfbe842a0 2024-05-27T08:39:43.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.608+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/3251877115 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef002a070 con 0x557dfbe842a0 2024-05-27T08:39:43.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.608+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/3251877115 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ef000fa60 con 0x557dfbe842a0 2024-05-27T08:39:43.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.608+0000 7f4f11c6cb80 10 client.15483 fetch_fsmap finished waiting for FSMap version 29 2024-05-27T08:39:43.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.609+0000 7f4f11c6cb80 10 client.15483 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:39:43.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.609+0000 7f4f11c6cb80 20 client.15483 populate_metadata read hostname 'smithi077' 2024-05-27T08:39:43.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.609+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x557dfbf08ba0 0x557dfbf0afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:39:43.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.609+0000 7f4f11c6cb80 4 client.15483 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "595a5120", "release": true, "await": true}] 2024-05-27T08:39:43.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.609+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/3251877115 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "set_id": "595a5120", "release": true, "await": true}) -- 0x557dfbde4ce0 con 0x557dfbf08ba0 2024-05-27T08:39:43.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.609+0000 7f4f05172700 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x557dfbf08ba0 0x557dfbf0afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:39:43.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.609+0000 7f4f05172700 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x557dfbf08ba0 0x557dfbf0afc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e48b53ff5c58bcf server_cookie=4d1301e88179dc5d in_seq=0 out_seq=0 2024-05-27T08:39:43.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.609+0000 7f4ee8ff9700 10 client.15483 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4ee8ff9700 1 -- 172.21.15.77:0/3251877115 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x557dfbde4ce0 con 0x557dfbf08ba0 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4ee8ff9700 10 client.15483 handle_command_reply: tid=0 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 client.15483 shutdown 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/3251877115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 msgr2=0x557dfbe95eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe95eb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0003760 tx=0x7f4ef000b720 comp rx=0 tx=0).stop 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/3251877115 shutdown_connections 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x557dfbf08ba0 0x557dfbf0afc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x557dfbe892b0 0x557dfbe92940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dfbe838e0 0x557dfbe95970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 --2- 172.21.15.77:0/3251877115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557dfbe842a0 0x557dfbe95eb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/3251877115 >> 172.21.15.77:0/3251877115 conn(0x557dfbdb1180 msgr2=0x557dfbe78410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:43.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4ee8ff9700 0 client.15483 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:39:43.614 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:39:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 29, 2024-05-27T08:39:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:39:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 4, 2024-05-27T08:39:43.614 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "595a5120": { 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 4, 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 149.4, 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:39:43.615 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/3251877115 shutdown_connections 2024-05-27T08:39:43.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 1 -- 172.21.15.77:0/3251877115 wait complete. 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.611+0000 7f4f11c6cb80 20 client.15483 trim_cache size 0 max 16384 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 -- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f4eec0b2b10 msgr2=0x7f4eec0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f4eec0b2b10 0x7f4eec0b4fd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005950 tx=0x7f4efc00b410 comp rx=0 tx=0).stop 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 -- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00108920 msgr2=0x7f4f0006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00108920 0x7f4f0006ddb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800ef30 tx=0x7f4ef800c3a0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 -- 172.21.15.77:0/4262579867 shutdown_connections 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f4eec0b2b10 0x7f4eec0b4fd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4f00108920 0x7f4f0006ddb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f00103850 0x7f4f00076d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 --2- 172.21.15.77:0/4262579867 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f00104200 0x7f4f000772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 -- 172.21.15.77:0/4262579867 >> 172.21.15.77:0/4262579867 conn(0x7f4f0007a5d0 msgr2=0x7f4f00101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:39:43.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 -- 172.21.15.77:0/4262579867 shutdown_connections 2024-05-27T08:39:43.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:39:43.612+0000 7f4f073d6700 1 -- 172.21.15.77:0/4262579867 wait complete. 2024-05-27T08:39:43.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:43 smithi179 ceph-mon[36481]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:39:43.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:43 smithi077 ceph-mon[32007]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:39:43.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:43 smithi092 ceph-mon[36049]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-27T08:39:44.423 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '595a5120', total seconds elapsed: 35.5 2024-05-27T08:39:45.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:45 smithi179 ceph-mon[36481]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:45.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:45 smithi077 ceph-mon[32007]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:45.988 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:45 smithi092 ceph-mon[36049]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:47.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:47 smithi179 ceph-mon[36481]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:47.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:47 smithi077 ceph-mon[32007]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:47.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:47 smithi092 ceph-mon[36049]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-27T08:39:49.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:49 smithi077 ceph-mon[32007]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-27T08:39:49.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:49 smithi077 ceph-mon[32007]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T08:39:49.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:49 smithi092 ceph-mon[36049]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-27T08:39:49.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:49 smithi092 ceph-mon[36049]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T08:39:50.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:49 smithi179 ceph-mon[36481]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 13 op/s 2024-05-27T08:39:50.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:49 smithi179 ceph-mon[36481]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-27T08:39:51.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:51 smithi077 ceph-mon[32007]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:51.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:51 smithi092 ceph-mon[36049]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:52.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:51 smithi179 ceph-mon[36481]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:53.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:53 smithi077 ceph-mon[32007]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-27T08:39:53.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:53 smithi092 ceph-mon[36049]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-27T08:39:54.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:53 smithi179 ceph-mon[36481]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-27T08:39:55.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:55 smithi077 ceph-mon[32007]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:55.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:55 smithi092 ceph-mon[36049]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:56.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:55 smithi179 ceph-mon[36481]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:57.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:57 smithi077 ceph-mon[32007]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:57.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:57 smithi092 ceph-mon[36049]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:58.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:57 smithi179 ceph-mon[36481]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 9 op/s 2024-05-27T08:39:59.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:39:59 smithi077 ceph-mon[32007]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-27T08:39:59.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:39:59 smithi092 ceph-mon[36049]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-27T08:40:00.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:39:59 smithi179 ceph-mon[36481]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 14 op/s 2024-05-27T08:40:00.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:00 smithi077 ceph-mon[32007]: overall HEALTH_OK 2024-05-27T08:40:00.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:00 smithi092 ceph-mon[36049]: overall HEALTH_OK 2024-05-27T08:40:01.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:00 smithi179 ceph-mon[36481]: overall HEALTH_OK 2024-05-27T08:40:01.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:01 smithi077 ceph-mon[32007]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-27T08:40:01.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:01 smithi092 ceph-mon[36049]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-27T08:40:02.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:01 smithi179 ceph-mon[36481]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-27T08:40:03.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:03 smithi092 ceph-mon[36049]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-27T08:40:04.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:03 smithi179 ceph-mon[36481]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-27T08:40:04.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:03 smithi077 ceph-mon[32007]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-05-27T08:40:05.500 INFO:tasks.workunit.client.0.smithi077.stderr:Note: switching to 'a9cb3a581a309a99b72997ae5ddb88084f5484c9'. 2024-05-27T08:40:05.500 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-05-27T08:40:05.500 INFO:tasks.workunit.client.0.smithi077.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-27T08:40:05.500 INFO:tasks.workunit.client.0.smithi077.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-27T08:40:05.500 INFO:tasks.workunit.client.0.smithi077.stderr:state without impacting any branches by switching back to a branch. 2024-05-27T08:40:05.500 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-05-27T08:40:05.500 INFO:tasks.workunit.client.0.smithi077.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-27T08:40:05.500 INFO:tasks.workunit.client.0.smithi077.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr: git switch -c 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr:Or undo this operation with: 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr: git switch - 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr: 2024-05-27T08:40:05.501 INFO:tasks.workunit.client.0.smithi077.stderr:HEAD is now at a9cb3a581a3 mds/quiesce: disable quiesce root debug parameters by default 2024-05-27T08:40:05.505 DEBUG:teuthology.orchestra.run.smithi077:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-27T08:40:05.538 INFO:tasks.workunit.client.0.smithi077.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-27T08:40:05.540 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T08:40:05.540 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-27T08:40:05.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:05 smithi092 ceph-mon[36049]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T08:40:06.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:05 smithi179 ceph-mon[36481]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T08:40:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:05 smithi077 ceph-mon[32007]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T08:40:06.209 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-27T08:40:06.272 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-27T08:40:06.314 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-27T08:40:06.315 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T08:40:06.315 INFO:tasks.workunit.client.0.smithi077.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-27T08:40:06.355 INFO:tasks.workunit.client.0.smithi077.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-27T08:40:06.359 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:40:06.359 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-27T08:40:06.415 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-05-27T08:40:06.416 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-27T08:40:06.416 DEBUG:teuthology.orchestra.run.smithi077:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a9cb3a581a309a99b72997ae5ddb88084f5484c9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-05-27T08:40:06.478 INFO:tasks.workunit.client.0.smithi077.stderr:+ dbench 1 2024-05-27T08:40:06.483 INFO:tasks.workunit.client.0.smithi077.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-27T08:40:06.483 INFO:tasks.workunit.client.0.smithi077.stdout: 2024-05-27T08:40:06.483 INFO:tasks.workunit.client.0.smithi077.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-27T08:40:06.485 INFO:tasks.workunit.client.0.smithi077.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-27T08:40:06.586 INFO:tasks.workunit.client.0.smithi077.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-27T08:40:06.586 INFO:tasks.workunit.client.0.smithi077.stdout:releasing clients 2024-05-27T08:40:07.585 INFO:tasks.workunit.client.0.smithi077.stdout: 1 3284 35.35 MB/sec warmup 1 sec latency 10.456 ms 2024-05-27T08:40:07.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:07 smithi092 ceph-mon[36049]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T08:40:08.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:07 smithi179 ceph-mon[36481]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T08:40:08.137 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:07 smithi077 ceph-mon[32007]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-27T08:40:08.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 6014 23.62 MB/sec warmup 2 sec latency 13.034 ms 2024-05-27T08:40:09.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 8527 21.42 MB/sec warmup 3 sec latency 12.098 ms 2024-05-27T08:40:09.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:09 smithi179 ceph-mon[36481]: pgmap v59: 97 pgs: 97 active+clean; 2.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 113 KiB/s wr, 40 op/s 2024-05-27T08:40:09.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:09 smithi092 ceph-mon[36049]: pgmap v59: 97 pgs: 97 active+clean; 2.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 113 KiB/s wr, 40 op/s 2024-05-27T08:40:10.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:09 smithi077 ceph-mon[32007]: pgmap v59: 97 pgs: 97 active+clean; 2.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 113 KiB/s wr, 40 op/s 2024-05-27T08:40:10.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 11388 19.67 MB/sec warmup 4 sec latency 11.393 ms 2024-05-27T08:40:11.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 14581 19.49 MB/sec warmup 5 sec latency 9.171 ms 2024-05-27T08:40:11.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:11 smithi092 ceph-mon[36049]: pgmap v60: 97 pgs: 97 active+clean; 2.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 113 KiB/s wr, 36 op/s 2024-05-27T08:40:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:11 smithi179 ceph-mon[36481]: pgmap v60: 97 pgs: 97 active+clean; 2.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 113 KiB/s wr, 36 op/s 2024-05-27T08:40:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:11 smithi077 ceph-mon[32007]: pgmap v60: 97 pgs: 97 active+clean; 2.2 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 113 KiB/s wr, 36 op/s 2024-05-27T08:40:12.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 17564 18.66 MB/sec warmup 6 sec latency 20.641 ms 2024-05-27T08:40:13.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 20238 17.68 MB/sec warmup 7 sec latency 12.343 ms 2024-05-27T08:40:13.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:13 smithi092 ceph-mon[36049]: pgmap v61: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 840 KiB/s wr, 122 op/s 2024-05-27T08:40:14.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:13 smithi179 ceph-mon[36481]: pgmap v61: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 840 KiB/s wr, 122 op/s 2024-05-27T08:40:14.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:13 smithi077 ceph-mon[32007]: pgmap v61: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 840 KiB/s wr, 122 op/s 2024-05-27T08:40:14.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 22734 17.60 MB/sec warmup 8 sec latency 19.628 ms 2024-05-27T08:40:15.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 25486 17.21 MB/sec warmup 9 sec latency 21.921 ms 2024-05-27T08:40:15.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:15 smithi092 ceph-mon[36049]: pgmap v62: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 818 KiB/s wr, 103 op/s 2024-05-27T08:40:16.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:15 smithi179 ceph-mon[36481]: pgmap v62: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 818 KiB/s wr, 103 op/s 2024-05-27T08:40:16.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:15 smithi077 ceph-mon[32007]: pgmap v62: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 818 KiB/s wr, 103 op/s 2024-05-27T08:40:16.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 28718 17.37 MB/sec warmup 10 sec latency 12.052 ms 2024-05-27T08:40:17.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 31756 17.13 MB/sec warmup 11 sec latency 12.308 ms 2024-05-27T08:40:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:17 smithi092 ceph-mon[36049]: pgmap v63: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 818 KiB/s wr, 103 op/s 2024-05-27T08:40:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:17 smithi179 ceph-mon[36481]: pgmap v63: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 818 KiB/s wr, 103 op/s 2024-05-27T08:40:18.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:17 smithi077 ceph-mon[32007]: pgmap v63: 97 pgs: 97 active+clean; 9.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 818 KiB/s wr, 103 op/s 2024-05-27T08:40:18.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 34365 16.68 MB/sec warmup 12 sec latency 15.967 ms 2024-05-27T08:40:19.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 36672 16.48 MB/sec warmup 13 sec latency 12.204 ms 2024-05-27T08:40:19.930 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:19 smithi077 ceph-mon[32007]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-05-27T08:40:19.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:19 smithi092 ceph-mon[36049]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-05-27T08:40:20.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:19 smithi179 ceph-mon[36481]: pgmap v64: 97 pgs: 97 active+clean; 16 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-05-27T08:40:20.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 39602 16.52 MB/sec warmup 14 sec latency 9.337 ms 2024-05-27T08:40:21.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 42793 16.65 MB/sec warmup 15 sec latency 21.120 ms 2024-05-27T08:40:21.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:21 smithi092 ceph-mon[36049]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-27T08:40:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:21 smithi179 ceph-mon[36481]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-27T08:40:22.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:21 smithi077 ceph-mon[32007]: pgmap v65: 97 pgs: 97 active+clean; 16 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-05-27T08:40:22.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 45476 16.36 MB/sec warmup 16 sec latency 21.640 ms 2024-05-27T08:40:23.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 47796 16.17 MB/sec warmup 17 sec latency 13.097 ms 2024-05-27T08:40:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:23 smithi092 ceph-mon[36049]: pgmap v66: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 255 op/s 2024-05-27T08:40:23.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:23 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:40:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:23 smithi179 ceph-mon[36481]: pgmap v66: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 255 op/s 2024-05-27T08:40:24.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:23 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:40:24.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:23 smithi077 ceph-mon[32007]: pgmap v66: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 255 op/s 2024-05-27T08:40:24.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:23 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:40:24.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 50350 16.04 MB/sec warmup 18 sec latency 16.429 ms 2024-05-27T08:40:24.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:24 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:40:24.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:24 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:40:24.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:24 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:40:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:24 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:40:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:24 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:40:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:24 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:40:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:24 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:40:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:24 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:40:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:24 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:40:25.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53242 16.11 MB/sec warmup 19 sec latency 125.111 ms 2024-05-27T08:40:25.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:25 smithi092 ceph-mon[36049]: pgmap v67: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-27T08:40:25.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:25 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:26.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:25 smithi179 ceph-mon[36481]: pgmap v67: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-27T08:40:26.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:25 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:26.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:25 smithi077 ceph-mon[32007]: pgmap v67: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-27T08:40:26.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:25 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:26.586 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53242 15.31 MB/sec warmup 20 sec latency 1125.241 ms 2024-05-27T08:40:27.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53242 14.58 MB/sec warmup 21 sec latency 2125.359 ms 2024-05-27T08:40:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:27 smithi092 ceph-mon[36049]: pgmap v68: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-27T08:40:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:27 smithi179 ceph-mon[36481]: pgmap v68: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-27T08:40:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:27 smithi077 ceph-mon[32007]: pgmap v68: 97 pgs: 97 active+clean; 22 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-05-27T08:40:28.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53323 13.93 MB/sec warmup 22 sec latency 2975.875 ms 2024-05-27T08:40:29.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53323 13.32 MB/sec warmup 23 sec latency 1105.968 ms 2024-05-27T08:40:29.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:29 smithi092 ceph-mon[36049]: pgmap v69: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-05-27T08:40:29.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:29 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:30.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:29 smithi179 ceph-mon[36481]: pgmap v69: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-05-27T08:40:30.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:29 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:29 smithi077 ceph-mon[32007]: pgmap v69: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 968 KiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-05-27T08:40:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:29 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:30.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53323 12.77 MB/sec warmup 24 sec latency 2106.017 ms 2024-05-27T08:40:31.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53323 12.25 MB/sec warmup 25 sec latency 3106.123 ms 2024-05-27T08:40:31.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:31 smithi092 ceph-mon[36049]: pgmap v70: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-05-27T08:40:32.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:31 smithi179 ceph-mon[36481]: pgmap v70: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-05-27T08:40:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:31 smithi077 ceph-mon[32007]: pgmap v70: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-05-27T08:40:32.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53323 11.78 MB/sec warmup 26 sec latency 4106.272 ms 2024-05-27T08:40:33.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53475 11.39 MB/sec warmup 27 sec latency 4954.834 ms 2024-05-27T08:40:33.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:33 smithi092 ceph-mon[36049]: pgmap v71: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-05-27T08:40:34.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:33 smithi179 ceph-mon[36481]: pgmap v71: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-05-27T08:40:34.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:33 smithi077 ceph-mon[32007]: pgmap v71: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-05-27T08:40:34.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53475 10.98 MB/sec warmup 28 sec latency 1002.675 ms 2024-05-27T08:40:35.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53475 10.60 MB/sec warmup 29 sec latency 2002.774 ms 2024-05-27T08:40:35.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:35 smithi092 ceph-mon[36049]: pgmap v72: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 459 KiB/s wr, 68 op/s 2024-05-27T08:40:36.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:35 smithi179 ceph-mon[36481]: pgmap v72: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 459 KiB/s wr, 68 op/s 2024-05-27T08:40:36.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:35 smithi077 ceph-mon[32007]: pgmap v72: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 459 KiB/s wr, 68 op/s 2024-05-27T08:40:36.587 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53475 10.25 MB/sec warmup 30 sec latency 3002.822 ms 2024-05-27T08:40:37.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53475 9.92 MB/sec warmup 31 sec latency 4002.977 ms 2024-05-27T08:40:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:37 smithi092 ceph-mon[36049]: pgmap v73: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 459 KiB/s wr, 68 op/s 2024-05-27T08:40:38.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:37 smithi179 ceph-mon[36481]: pgmap v73: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 459 KiB/s wr, 68 op/s 2024-05-27T08:40:38.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:37 smithi077 ceph-mon[32007]: pgmap v73: 97 pgs: 97 active+clean; 26 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 459 KiB/s wr, 68 op/s 2024-05-27T08:40:38.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53530 9.61 MB/sec warmup 32 sec latency 4551.587 ms 2024-05-27T08:40:39.024 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.6 2024-05-27T08:40:39.025 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:40:39.269 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:40:39.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53622 9.33 MB/sec warmup 33 sec latency 887.417 ms 2024-05-27T08:40:39.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:39 smithi092 ceph-mon[36049]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 470 KiB/s wr, 76 op/s 2024-05-27T08:40:40.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:39 smithi179 ceph-mon[36481]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 470 KiB/s wr, 76 op/s 2024-05-27T08:40:40.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:39 smithi077 ceph-mon[32007]: pgmap v74: 97 pgs: 97 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 470 KiB/s wr, 76 op/s 2024-05-27T08:40:40.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53622 9.05 MB/sec warmup 34 sec latency 1887.500 ms 2024-05-27T08:40:40.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 -- 172.21.15.77:0/176884454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa0103590 msgr2=0x7fdaa0103a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:40.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/176884454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa0103590 0x7fdaa0103a10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fda94009990 tx=0x7fda94009ca0 comp rx=0 tx=0).stop 2024-05-27T08:40:40.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 -- 172.21.15.77:0/176884454 shutdown_connections 2024-05-27T08:40:40.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/176884454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa0103f50 0x7fdaa0110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:40.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/176884454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa01096a0 0x7fdaa0109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:40.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/176884454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa0103590 0x7fdaa0103a10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:40.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 -- 172.21.15.77:0/176884454 >> 172.21.15.77:0/176884454 conn(0x7fdaa007a6f0 msgr2=0x7fdaa007ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:40:40.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 -- 172.21.15.77:0/176884454 shutdown_connections 2024-05-27T08:40:40.990 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.988+0000 7fdaa7d92700 1 -- 172.21.15.77:0/176884454 wait complete. 2024-05-27T08:40:40.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.989+0000 7fdaa7d92700 1 Processor -- start 2024-05-27T08:40:40.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.989+0000 7fdaa7d92700 1 -- start start 2024-05-27T08:40:40.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa7d92700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa0103590 0x7fdaa019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:40.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa7d92700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa0103f50 0x7fdaa019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:40.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa7d92700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa01096a0 0x7fdaa0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa7d92700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa01137f0 con 0x7fdaa01096a0 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa7d92700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0113670 con 0x7fdaa0103f50 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa7d92700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0113970 con 0x7fdaa0103590 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa01096a0 0x7fdaa0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa01096a0 0x7fdaa0199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46944/0 (socket says 172.21.15.77:46944) 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 -- 172.21.15.77:0/2088998734 learned_addr learned my addr 172.21.15.77:0/2088998734 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa532d700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa0103f50 0x7fdaa019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa5b2e700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa0103590 0x7fdaa019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 -- 172.21.15.77:0/2088998734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa0103590 msgr2=0x7fdaa019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa0103590 0x7fdaa019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 -- 172.21.15.77:0/2088998734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa0103f50 msgr2=0x7fdaa019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa0103f50 0x7fdaa019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:40.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 -- 172.21.15.77:0/2088998734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa019a290 con 0x7fdaa01096a0 2024-05-27T08:40:40.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.990+0000 7fdaa632f700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa01096a0 0x7fdaa0199b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b9e0 tx=0x7fda9000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:40:40.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.991+0000 7fda9effd700 1 -- 172.21.15.77:0/2088998734 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9000d820 con 0x7fdaa01096a0 2024-05-27T08:40:40.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.991+0000 7fda9effd700 1 -- 172.21.15.77:0/2088998734 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda90015420 con 0x7fdaa01096a0 2024-05-27T08:40:40.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.991+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa019a580 con 0x7fdaa01096a0 2024-05-27T08:40:40.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.991+0000 7fda9effd700 1 -- 172.21.15.77:0/2088998734 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90014610 con 0x7fdaa01096a0 2024-05-27T08:40:40.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.991+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa01a67f0 con 0x7fdaa01096a0 2024-05-27T08:40:40.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.992+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa006a230 con 0x7fdaa01096a0 2024-05-27T08:40:40.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.993+0000 7fda9effd700 1 -- 172.21.15.77:0/2088998734 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fda9000d9c0 con 0x7fdaa01096a0 2024-05-27T08:40:40.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.994+0000 7fda9effd700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fda880b29f0 0x7fda880b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:40.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.994+0000 7fdaa5b2e700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fda880b29f0 0x7fda880b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:40.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.994+0000 7fda9effd700 1 -- 172.21.15.77:0/2088998734 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fda900e2320 con 0x7fdaa01096a0 2024-05-27T08:40:40.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.994+0000 7fdaa5b2e700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fda880b29f0 0x7fda880b4eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda98009a10 tx=0x7fda98009190 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:40:40.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:40.997+0000 7fda9effd700 1 -- 172.21.15.77:0/2088998734 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda900a9d10 con 0x7fdaa01096a0 2024-05-27T08:40:41.253 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.252+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaa019afd0 con 0x7fdaa01096a0 2024-05-27T08:40:41.254 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.253+0000 7fda9effd700 1 -- 172.21.15.77:0/2088998734 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11934 (secure 0 0 0) 0x7fda900a9610 con 0x7fdaa01096a0 2024-05-27T08:40:41.254 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:40:41.254 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:40:28.664921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24418,24434,24410,24412,24433]},"id":1}]} 2024-05-27T08:40:41.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fda880b29f0 msgr2=0x7fda880b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:41.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fda880b29f0 0x7fda880b4eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda98009a10 tx=0x7fda98009190 comp rx=0 tx=0).stop 2024-05-27T08:40:41.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa01096a0 msgr2=0x7fdaa0199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:41.256 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa01096a0 0x7fdaa0199b30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fda9000b9e0 tx=0x7fda9000bcf0 comp rx=0 tx=0).stop 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 shutdown_connections 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fda880b29f0 0x7fda880b4eb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa01096a0 0x7fdaa0199b30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaa0103590 0x7fdaa019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 --2- 172.21.15.77:0/2088998734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdaa0103f50 0x7fdaa019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 >> 172.21.15.77:0/2088998734 conn(0x7fdaa007a6f0 msgr2=0x7fdaa01021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 shutdown_connections 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:41.255+0000 7fdaa7d92700 1 -- 172.21.15.77:0/2088998734 wait complete. 2024-05-27T08:40:41.257 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 31 2024-05-27T08:40:41.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53622 8.79 MB/sec warmup 35 sec latency 2887.569 ms 2024-05-27T08:40:41.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:41 smithi077 ceph-mon[32007]: pgmap v75: 97 pgs: 97 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-27T08:40:41.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:41 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2088998734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:40:41.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:41 smithi092 ceph-mon[36049]: pgmap v75: 97 pgs: 97 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-27T08:40:41.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:41 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2088998734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:40:42.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:41 smithi179 ceph-mon[36481]: pgmap v75: 97 pgs: 97 active+clean; 26 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-05-27T08:40:42.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:41 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2088998734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:40:42.145 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 90 --expiration 154.6 --await-for 45.0' 2024-05-27T08:40:42.145 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db / --timeout 90 --expiration 154.6 --await-for 45.0 2024-05-27T08:40:42.398 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:40:42.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53622 8.55 MB/sec warmup 36 sec latency 3887.609 ms 2024-05-27T08:40:43.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 53712 8.32 MB/sec warmup 37 sec latency 4738.893 ms 2024-05-27T08:40:43.961 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 -- 172.21.15.77:0/3577503764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 msgr2=0x7fb438108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:43.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 --2- 172.21.15.77:0/3577503764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 0x7fb438108980 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb4280099c0 tx=0x7fb428009cd0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 -- 172.21.15.77:0/3577503764 shutdown_connections 2024-05-27T08:40:43.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 --2- 172.21.15.77:0/3577503764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb438103fb0 0x7fb4381104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 --2- 172.21.15.77:0/3577503764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 0x7fb438108980 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 --2- 172.21.15.77:0/3577503764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4381035f0 0x7fb438103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 -- 172.21.15.77:0/3577503764 >> 172.21.15.77:0/3577503764 conn(0x7fb43807a6f0 msgr2=0x7fb43807ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:40:43.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 -- 172.21.15.77:0/3577503764 shutdown_connections 2024-05-27T08:40:43.962 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.958+0000 7fb43d457700 1 -- 172.21.15.77:0/3577503764 wait complete. 2024-05-27T08:40:43.962 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:43 smithi077 ceph-mon[32007]: pgmap v76: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.3 MiB/s wr, 155 op/s 2024-05-27T08:40:43.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb43d457700 1 Processor -- start 2024-05-27T08:40:43.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb43d457700 1 -- start start 2024-05-27T08:40:43.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb43d457700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4381035f0 0x7fb438075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb43d457700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103fb0 0x7fb438075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb43d457700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 0x7fb4380762c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb43d457700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381138a0 con 0x7fb4381085a0 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb43d457700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb438113720 con 0x7fb4381035f0 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb43d457700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb438113a20 con 0x7fb438103fb0 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.959+0000 7fb4377fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 0x7fb4380762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb4377fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 0x7fb4380762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46960/0 (socket says 172.21.15.77:46960) 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb4377fe700 1 -- 172.21.15.77:0/443072123 learned_addr learned my addr 172.21.15.77:0/443072123 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb436ffd700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4381035f0 0x7fb438075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103fb0 0x7fb438075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb4377fe700 1 -- 172.21.15.77:0/443072123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103fb0 msgr2=0x7fb438075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb4377fe700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103fb0 0x7fb438075d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb4377fe700 1 -- 172.21.15.77:0/443072123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4381035f0 msgr2=0x7fb438075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:43.965 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb4377fe700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4381035f0 0x7fb438075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb4377fe700 1 -- 172.21.15.77:0/443072123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4381a81d0 con 0x7fb4381085a0 2024-05-27T08:40:43.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb4377fe700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 0x7fb4380762c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb43000b970 tx=0x7fb43000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:40:43.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb434ff9700 1 -- 172.21.15.77:0/443072123 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43000d860 con 0x7fb4381085a0 2024-05-27T08:40:43.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb434ff9700 1 -- 172.21.15.77:0/443072123 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb43000e430 con 0x7fb4381085a0 2024-05-27T08:40:43.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb43d457700 1 -- 172.21.15.77:0/443072123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4381a84c0 con 0x7fb4381085a0 2024-05-27T08:40:43.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb434ff9700 1 -- 172.21.15.77:0/443072123 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430017660 con 0x7fb4381085a0 2024-05-27T08:40:43.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.960+0000 7fb43d457700 1 -- 172.21.15.77:0/443072123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4381a8a00 con 0x7fb4381085a0 2024-05-27T08:40:43.966 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.963+0000 7fb434ff9700 1 -- 172.21.15.77:0/443072123 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb4300203f0 con 0x7fb4381085a0 2024-05-27T08:40:43.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.964+0000 7fb434ff9700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb4240b2b20 0x7fb4240b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.967 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.964+0000 7fb436ffd700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb4240b2b20 0x7fb4240b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.964+0000 7fb434ff9700 1 -- 172.21.15.77:0/443072123 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb4300e2140 con 0x7fb4381085a0 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb447cedb80 1 Processor -- start 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb447cedb80 1 -- start start 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb436ffd700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb4240b2b20 0x7fb4240b4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb428006010 tx=0x7fb4280096b0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815220720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x56481521adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 0x564815227170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb447cedb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564815179e20 con 0x56481521a970 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb447cedb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5648151699c0 con 0x564815220340 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb447cedb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56481516da00 con 0x56481521b330 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x56481521adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x56481521adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46966/0 (socket says 172.21.15.77:46966) 2024-05-27T08:40:43.968 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb42e7fc700 1 -- 172.21.15.77:0/2248291666 learned_addr learned my addr 172.21.15.77:0/2248291666 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:40:43.969 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb4377fe700 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 0x564815227170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.978 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb436ffd700 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815220720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb42e7fc700 1 -- 172.21.15.77:0/2248291666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 msgr2=0x564815227170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:40:43.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 0x564815227170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb42e7fc700 1 -- 172.21.15.77:0/2248291666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 msgr2=0x564815220720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:40:43.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815220720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.979 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb42e7fc700 1 -- 172.21.15.77:0/2248291666 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648150b9a60 con 0x56481521a970 2024-05-27T08:40:43.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.965+0000 7fb4377fe700 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 0x564815227170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:40:43.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.966+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x56481521adf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb420009a80 tx=0x7fb420009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00e29eb638f067a server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:40:43.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.966+0000 7fb42d7fa700 1 -- 172.21.15.77:0/2248291666 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420017070 con 0x56481521a970 2024-05-27T08:40:43.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.966+0000 7fb42d7fa700 1 -- 172.21.15.77:0/2248291666 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb42000ed20 con 0x56481521a970 2024-05-27T08:40:43.980 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.966+0000 7fb42d7fa700 1 -- 172.21.15.77:0/2248291666 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420013d10 con 0x56481521a970 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- 172.21.15.77:0/2248291666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 msgr2=0x56481521adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x56481521adf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb420009a80 tx=0x7fb420009d90 comp rx=0 tx=0).stop 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- 172.21.15.77:0/2248291666 shutdown_connections 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 0x564815227170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815220720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2248291666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x56481521adf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- 172.21.15.77:0/2248291666 >> 172.21.15.77:0/2248291666 conn(0x564815148210 msgr2=0x564815215280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- 172.21.15.77:0/2248291666 shutdown_connections 2024-05-27T08:40:43.984 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- 172.21.15.77:0/2248291666 wait complete. 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- start start 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 Processor -- start 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521a970 0x564815226e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x564815224110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815224650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564815070130 con 0x56481521b330 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x56481510b950 con 0x564815220340 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564815227770 con 0x56481521a970 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.967+0000 7fb447cedb80 10 client.0 _ll_register_callbacks cb 0x564815104ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x564815224110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x564815224110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46968/0 (socket says 172.21.15.77:46968) 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 -- 172.21.15.77:0/3026925587 learned_addr learned my addr 172.21.15.77:0/3026925587 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:40:43.985 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb436ffd700 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521a970 0x564815226e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb4377fe700 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815224650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 -- 172.21.15.77:0/3026925587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521a970 msgr2=0x564815226e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521a970 0x564815226e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 -- 172.21.15.77:0/3026925587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 msgr2=0x564815224650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815224650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 -- 172.21.15.77:0/3026925587 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564815108820 con 0x56481521b330 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x564815224110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb42000ece0 tx=0x7fb420013740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f8ce538382bfe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb41e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb41e7fc700 1 -- 172.21.15.77:0/3026925587 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420029060 con 0x56481521b330 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb447cedb80 1 -- 172.21.15.77:0/3026925587 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42000ed20 con 0x56481521b330 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb41e7fc700 1 -- 172.21.15.77:0/3026925587 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb420014c30 con 0x56481521b330 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb41e7fc700 1 -- 172.21.15.77:0/3026925587 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb420017040 con 0x56481521b330 2024-05-27T08:40:43.986 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb447cedb80 10 client.15501 fetch_fsmap learned FSMap version 31 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.968+0000 7fb447cedb80 1 -- 172.21.15.77:0/3026925587 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564815231e80 con 0x56481521b330 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb41e7fc700 1 -- 172.21.15.77:0/3026925587 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb42000ed20 con 0x56481521b330 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb41e7fc700 1 -- 172.21.15.77:0/3026925587 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fb42002c3a0 con 0x56481521b330 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb447cedb80 10 client.15501 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb447cedb80 10 client.15501 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb447cedb80 20 client.15501 populate_metadata read hostname 'smithi077' 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb447cedb80 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564815234350 0x564815236740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb447cedb80 4 client.15501 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb447cedb80 1 -- 172.21.15.77:0/3026925587 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648151e6b70 con 0x564815234350 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.969+0000 7fb436ffd700 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564815234350 0x564815236740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.970+0000 7fb41e7fc700 10 client.15501 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.970+0000 7fb436ffd700 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564815234350 0x564815236740 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adab29e1c9eb80a9 server_cookie=e3448870c3a4bd46 in_seq=0 out_seq=0 2024-05-27T08:40:43.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.970+0000 7fb41e7fc700 1 -- 172.21.15.77:0/3026925587 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5648151e6b70 con 0x564815234350 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.970+0000 7fb41e7fc700 10 client.15501 handle_command_reply: tid=0 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.970+0000 7fb447cedb80 1 client.15501 shutdown 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.970+0000 7fb447cedb80 1 -- 172.21.15.77:0/3026925587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 msgr2=0x564815224110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x564815224110 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb42000ece0 tx=0x7fb420013740 comp rx=0 tx=0).stop 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 -- 172.21.15.77:0/3026925587 shutdown_connections 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564815234350 0x564815236740 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815224650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521a970 0x564815226e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb41e7fc700 0 client.15501 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 --2- 172.21.15.77:0/3026925587 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x564815224110 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 -- 172.21.15.77:0/3026925587 >> 172.21.15.77:0/3026925587 conn(0x564815148210 msgr2=0x56481520fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 -- 172.21.15.77:0/3026925587 shutdown_connections 2024-05-27T08:40:43.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 1 -- 172.21.15.77:0/3026925587 wait complete. 2024-05-27T08:40:43.989 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.971+0000 7fb447cedb80 20 client.15501 trim_cache size 0 max 16384 2024-05-27T08:40:43.989 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:43 smithi092 ceph-mon[36049]: pgmap v76: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.3 MiB/s wr, 155 op/s 2024-05-27T08:40:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb447cedb80 1 Processor -- start 2024-05-27T08:40:43.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb447cedb80 1 -- start start 2024-05-27T08:40:43.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56481521a970 0x564815226050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x56481521ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564815220340 0x56481521d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb447cedb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381aff70 con 0x56481521b330 2024-05-27T08:40:43.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb447cedb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381b1b70 con 0x56481521a970 2024-05-27T08:40:43.993 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb447cedb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381b0640 con 0x564815220340 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x56481521ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x56481521ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46980/0 (socket says 172.21.15.77:46980) 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 -- 172.21.15.77:0/340825234 learned_addr learned my addr 172.21.15.77:0/340825234 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb436ffd700 1 --2- 172.21.15.77:0/340825234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56481521a970 0x564815226050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 -- 172.21.15.77:0/340825234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564815220340 msgr2=0x56481521d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/340825234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564815220340 0x56481521d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 -- 172.21.15.77:0/340825234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56481521a970 msgr2=0x564815226050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/340825234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56481521a970 0x564815226050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 -- 172.21.15.77:0/340825234 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481520d930 con 0x56481521b330 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.991+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/340825234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x56481521ce20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb42000e4b0 tx=0x7fb420009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a417883f7988965d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb42cff9700 1 -- 172.21.15.77:0/340825234 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420029040 con 0x56481521b330 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb42cff9700 1 -- 172.21.15.77:0/340825234 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb42002c3a0 con 0x56481521b330 2024-05-27T08:40:43.994 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb42cff9700 1 -- 172.21.15.77:0/340825234 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb420025bf0 con 0x56481521b330 2024-05-27T08:40:43.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 -- 172.21.15.77:0/340825234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 msgr2=0x56481521ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:43.995 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 --2- 172.21.15.77:0/340825234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x56481521ce20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb42000e4b0 tx=0x7fb420009d90 comp rx=0 tx=0).stop 2024-05-27T08:40:43.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 -- 172.21.15.77:0/340825234 shutdown_connections 2024-05-27T08:40:43.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 --2- 172.21.15.77:0/340825234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564815220340 0x56481521d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 --2- 172.21.15.77:0/340825234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56481521a970 0x564815226050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.997 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 --2- 172.21.15.77:0/340825234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521b330 0x56481521ce20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 -- 172.21.15.77:0/340825234 >> 172.21.15.77:0/340825234 conn(0x564815148210 msgr2=0x564815225940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 -- 172.21.15.77:0/340825234 shutdown_connections 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.992+0000 7fb447cedb80 1 -- 172.21.15.77:0/340825234 wait complete. 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 1 -- start start 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 1 Processor -- start 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x5648152279c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 0x564815227f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815228440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381b13e0 con 0x56481521a970 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381b0d10 con 0x564815220340 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb4381adf10 con 0x56481521b330 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb447cedb80 10 client.0 _ll_register_callbacks cb 0x564815104ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:40:43.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x5648152279c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb436ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x5648152279c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46986/0 (socket says 172.21.15.77:46986) 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb436ffd700 1 -- 172.21.15.77:0/2036501037 learned_addr learned my addr 172.21.15.77:0/2036501037 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb4377fe700 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815228440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb436ffd700 1 -- 172.21.15.77:0/2036501037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 msgr2=0x564815227f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb436ffd700 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 0x564815227f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb436ffd700 1 -- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 msgr2=0x564815228440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb436ffd700 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815228440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.993+0000 7fb436ffd700 1 -- 172.21.15.77:0/2036501037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481529ef20 con 0x56481521a970 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb4377fe700 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815228440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb436ffd700 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x5648152279c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb428000c00 tx=0x7fb428009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d155b60b5fa630 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb41e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb41e7fc700 1 -- 172.21.15.77:0/2036501037 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428024c70 con 0x56481521a970 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 1 -- 172.21.15.77:0/2036501037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42002c3a0 con 0x56481521a970 2024-05-27T08:40:43.999 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb41e7fc700 1 -- 172.21.15.77:0/2036501037 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb42803b480 con 0x56481521a970 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb41e7fc700 1 -- 172.21.15.77:0/2036501037 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb428024e10 con 0x56481521a970 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 10 client.15507 fetch_fsmap learned FSMap version 31 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 1 -- 172.21.15.77:0/2036501037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5648152a0240 con 0x56481521a970 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb41e7fc700 1 -- 172.21.15.77:0/2036501037 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb42002c3a0 con 0x56481521a970 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb41e7fc700 1 -- 172.21.15.77:0/2036501037 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 31) ==== 8370+0+0 (secure 0 0 0) 0x7fb42800f750 con 0x56481521a970 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 10 client.15507 fetch_fsmap finished waiting for FSMap version 31 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 10 client.15507 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 20 client.15507 populate_metadata read hostname 'smithi077' 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5648152a0a00 0x5648152a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 4 client.15507 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.6, "await_for": 45.0}] 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.994+0000 7fb447cedb80 1 -- 172.21.15.77:0/2036501037 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.6, "await_for": 45.0}) -- 0x56481517bd70 con 0x5648152a0a00 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.995+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5648152a0a00 0x5648152a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.995+0000 7fb42e7fc700 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5648152a0a00 0x5648152a2e40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bbd78a4f981accd server_cookie=28da793f1b32f66a in_seq=0 out_seq=0 2024-05-27T08:40:44.000 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:43.995+0000 7fb41e7fc700 10 client.15507 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:40:44.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb41e7fc700 1 -- 172.21.15.77:0/2036501037 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56481517bd70 con 0x5648152a0a00 2024-05-27T08:40:44.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb41e7fc700 10 client.15507 handle_command_reply: tid=0 2024-05-27T08:40:44.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 client.15507 shutdown 2024-05-27T08:40:44.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 -- 172.21.15.77:0/2036501037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 msgr2=0x5648152279c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:44.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x5648152279c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb428000c00 tx=0x7fb428009cd0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.055 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 -- 172.21.15.77:0/2036501037 shutdown_connections 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5648152a0a00 0x5648152a2e40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564815220340 0x564815228440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb41e7fc700 0 client.15507 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56481521a970 0x5648152279c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 --2- 172.21.15.77:0/2036501037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56481521b330 0x564815227f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 -- 172.21.15.77:0/2036501037 >> 172.21.15.77:0/2036501037 conn(0x564815148210 msgr2=0x56481520fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 -- 172.21.15.77:0/2036501037 shutdown_connections 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 1 -- 172.21.15.77:0/2036501037 wait complete. 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.053+0000 7fb447cedb80 20 client.15507 trim_cache size 0 max 16384 2024-05-27T08:40:44.056 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 31, 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 6, 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "849bc6b8": { 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 6, 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:40:44.057 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 154.6, 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:40:44.058 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:40:44.059 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:40:44.059 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:40:44.059 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 -- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb4240b2b20 msgr2=0x7fb4240b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb4240b2b20 0x7fb4240b4fe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb428006010 tx=0x7fb4280096b0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 -- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 msgr2=0x7fb4380762c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 0x7fb4380762c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb43000b970 tx=0x7fb43000bc80 comp rx=0 tx=0).stop 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 -- 172.21.15.77:0/443072123 shutdown_connections 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb4240b2b20 0x7fb4240b4fe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4381085a0 0x7fb4380762c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4381035f0 0x7fb438075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 --2- 172.21.15.77:0/443072123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103fb0 0x7fb438075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 -- 172.21.15.77:0/443072123 >> 172.21.15.77:0/443072123 conn(0x7fb43807a6f0 msgr2=0x7fb4381015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 -- 172.21.15.77:0/443072123 shutdown_connections 2024-05-27T08:40:44.060 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:40:44.054+0000 7fb43d457700 1 -- 172.21.15.77:0/443072123 wait complete. 2024-05-27T08:40:44.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:43 smithi179 ceph-mon[36481]: pgmap v76: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.3 MiB/s wr, 155 op/s 2024-05-27T08:40:44.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 8.12 MB/sec warmup 38 sec latency 719.575 ms 2024-05-27T08:40:44.857 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '849bc6b8', quiesce took 5.8 seconds. Will release after: 28.8 2024-05-27T08:40:45.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 7.91 MB/sec warmup 39 sec latency 1719.654 ms 2024-05-27T08:40:45.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:45 smithi092 ceph-mon[36049]: pgmap v77: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.3 MiB/s wr, 146 op/s 2024-05-27T08:40:46.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:45 smithi179 ceph-mon[36481]: pgmap v77: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.3 MiB/s wr, 146 op/s 2024-05-27T08:40:46.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:45 smithi077 ceph-mon[32007]: pgmap v77: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.3 MiB/s wr, 146 op/s 2024-05-27T08:40:46.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 7.71 MB/sec warmup 40 sec latency 2719.745 ms 2024-05-27T08:40:47.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 7.53 MB/sec warmup 41 sec latency 3719.847 ms 2024-05-27T08:40:47.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:47 smithi092 ceph-mon[36049]: pgmap v78: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.3 MiB/s wr, 146 op/s 2024-05-27T08:40:48.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:47 smithi179 ceph-mon[36481]: pgmap v78: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.3 MiB/s wr, 146 op/s 2024-05-27T08:40:48.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:47 smithi077 ceph-mon[32007]: pgmap v78: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 5.3 MiB/s wr, 146 op/s 2024-05-27T08:40:48.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 7.35 MB/sec warmup 42 sec latency 4719.963 ms 2024-05-27T08:40:49.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 7.17 MB/sec warmup 43 sec latency 5720.083 ms 2024-05-27T08:40:49.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:49 smithi092 ceph-mon[36049]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.3 MiB/s wr, 165 op/s 2024-05-27T08:40:49.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:49 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:50.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:49 smithi179 ceph-mon[36481]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.3 MiB/s wr, 165 op/s 2024-05-27T08:40:50.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:49 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:50.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:49 smithi077 ceph-mon[32007]: pgmap v79: 97 pgs: 97 active+clean; 43 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.3 MiB/s wr, 165 op/s 2024-05-27T08:40:50.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:49 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:40:50.588 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 7.01 MB/sec warmup 44 sec latency 6720.182 ms 2024-05-27T08:40:51.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 6.86 MB/sec warmup 45 sec latency 7720.299 ms 2024-05-27T08:40:51.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:51 smithi092 ceph-mon[36049]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.3 MiB/s wr, 157 op/s 2024-05-27T08:40:52.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:51 smithi179 ceph-mon[36481]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.3 MiB/s wr, 157 op/s 2024-05-27T08:40:52.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:51 smithi077 ceph-mon[32007]: pgmap v80: 97 pgs: 97 active+clean; 43 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.3 MiB/s wr, 157 op/s 2024-05-27T08:40:52.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 6.71 MB/sec warmup 46 sec latency 8720.396 ms 2024-05-27T08:40:53.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 6.56 MB/sec warmup 47 sec latency 9720.472 ms 2024-05-27T08:40:53.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:53 smithi092 ceph-mon[36049]: pgmap v81: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 5.3 MiB/s wr, 168 op/s 2024-05-27T08:40:54.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:53 smithi179 ceph-mon[36481]: pgmap v81: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 5.3 MiB/s wr, 168 op/s 2024-05-27T08:40:54.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:53 smithi077 ceph-mon[32007]: pgmap v81: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 5.3 MiB/s wr, 168 op/s 2024-05-27T08:40:54.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 6.43 MB/sec warmup 48 sec latency 10720.568 ms 2024-05-27T08:40:55.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 6.30 MB/sec warmup 49 sec latency 11720.623 ms 2024-05-27T08:40:55.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:55 smithi092 ceph-mon[36049]: pgmap v82: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-05-27T08:40:56.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:55 smithi179 ceph-mon[36481]: pgmap v82: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-05-27T08:40:56.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:55 smithi077 ceph-mon[32007]: pgmap v82: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-05-27T08:40:56.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 6.17 MB/sec warmup 50 sec latency 12720.707 ms 2024-05-27T08:40:57.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 6.05 MB/sec warmup 51 sec latency 13720.806 ms 2024-05-27T08:40:57.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:57 smithi092 ceph-mon[36049]: pgmap v83: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-05-27T08:40:58.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:57 smithi179 ceph-mon[36481]: pgmap v83: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-05-27T08:40:58.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:57 smithi077 ceph-mon[32007]: pgmap v83: 97 pgs: 97 active+clean; 40 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-05-27T08:40:58.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.93 MB/sec warmup 52 sec latency 14720.940 ms 2024-05-27T08:40:59.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.82 MB/sec warmup 53 sec latency 15721.022 ms 2024-05-27T08:40:59.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:40:59 smithi092 ceph-mon[36049]: pgmap v84: 97 pgs: 97 active+clean; 41 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-05-27T08:41:00.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:40:59 smithi179 ceph-mon[36481]: pgmap v84: 97 pgs: 97 active+clean; 41 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-05-27T08:41:00.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:40:59 smithi077 ceph-mon[32007]: pgmap v84: 97 pgs: 97 active+clean; 41 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-05-27T08:41:00.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.71 MB/sec warmup 54 sec latency 16721.065 ms 2024-05-27T08:41:01.589 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.61 MB/sec warmup 55 sec latency 17721.210 ms 2024-05-27T08:41:01.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:01 smithi092 ceph-mon[36049]: pgmap v85: 97 pgs: 97 active+clean; 41 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-05-27T08:41:02.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:01 smithi179 ceph-mon[36481]: pgmap v85: 97 pgs: 97 active+clean; 41 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-05-27T08:41:02.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:01 smithi077 ceph-mon[32007]: pgmap v85: 97 pgs: 97 active+clean; 41 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-05-27T08:41:02.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.51 MB/sec warmup 56 sec latency 18721.295 ms 2024-05-27T08:41:03.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.41 MB/sec warmup 57 sec latency 19721.432 ms 2024-05-27T08:41:03.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:03 smithi092 ceph-mon[36049]: pgmap v86: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 KiB/s wr, 36 op/s 2024-05-27T08:41:04.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:03 smithi179 ceph-mon[36481]: pgmap v86: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 KiB/s wr, 36 op/s 2024-05-27T08:41:04.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:03 smithi077 ceph-mon[32007]: pgmap v86: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 33 KiB/s wr, 36 op/s 2024-05-27T08:41:04.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.32 MB/sec warmup 58 sec latency 20721.532 ms 2024-05-27T08:41:05.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.23 MB/sec warmup 59 sec latency 21721.649 ms 2024-05-27T08:41:06.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:05 smithi179 ceph-mon[36481]: pgmap v87: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-27T08:41:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:05 smithi077 ceph-mon[32007]: pgmap v87: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-27T08:41:06.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:05 smithi092 ceph-mon[36049]: pgmap v87: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-27T08:41:06.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.14 MB/sec warmup 60 sec latency 22721.751 ms 2024-05-27T08:41:07.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 5.06 MB/sec warmup 61 sec latency 23721.838 ms 2024-05-27T08:41:08.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:07 smithi179 ceph-mon[36481]: pgmap v88: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-27T08:41:08.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:07 smithi077 ceph-mon[32007]: pgmap v88: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-27T08:41:08.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:07 smithi092 ceph-mon[36049]: pgmap v88: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-05-27T08:41:08.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.98 MB/sec warmup 62 sec latency 24721.952 ms 2024-05-27T08:41:09.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.90 MB/sec warmup 63 sec latency 25722.041 ms 2024-05-27T08:41:10.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:09 smithi179 ceph-mon[36481]: pgmap v89: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-05-27T08:41:10.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:09 smithi077 ceph-mon[32007]: pgmap v89: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-05-27T08:41:10.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:09 smithi092 ceph-mon[36049]: pgmap v89: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-05-27T08:41:10.590 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.82 MB/sec warmup 64 sec latency 26722.142 ms 2024-05-27T08:41:11.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.75 MB/sec warmup 65 sec latency 27722.234 ms 2024-05-27T08:41:12.037 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:11 smithi092 ceph-mon[36049]: pgmap v90: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-05-27T08:41:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:11 smithi179 ceph-mon[36481]: pgmap v90: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-05-27T08:41:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:11 smithi077 ceph-mon[32007]: pgmap v90: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-05-27T08:41:12.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.67 MB/sec warmup 66 sec latency 28722.315 ms 2024-05-27T08:41:13.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.60 MB/sec warmup 67 sec latency 29722.407 ms 2024-05-27T08:41:13.658 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:41:13.914 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:41:14.036 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:13 smithi077 ceph-mon[32007]: pgmap v91: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 29 op/s 2024-05-27T08:41:14.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:13 smithi179 ceph-mon[36481]: pgmap v91: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 29 op/s 2024-05-27T08:41:14.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:13 smithi092 ceph-mon[36049]: pgmap v91: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 50 KiB/s wr, 29 op/s 2024-05-27T08:41:14.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.54 MB/sec warmup 68 sec latency 30722.525 ms 2024-05-27T08:41:15.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.47 MB/sec warmup 69 sec latency 31722.588 ms 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 -- 172.21.15.77:0/746763181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc0102470 msgr2=0x7f5bc01028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/746763181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc0102470 0x7f5bc01028f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009990 tx=0x7f5bb4009ca0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 -- 172.21.15.77:0/746763181 shutdown_connections 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/746763181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bc0102e30 0x7f5bc010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/746763181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bc01084b0 0x7f5bc0108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/746763181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc0102470 0x7f5bc01028f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 -- 172.21.15.77:0/746763181 >> 172.21.15.77:0/746763181 conn(0x7f5bc0079410 msgr2=0x7f5bc0079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 -- 172.21.15.77:0/746763181 shutdown_connections 2024-05-27T08:41:15.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.611+0000 7f5bc49e4700 1 -- 172.21.15.77:0/746763181 wait complete. 2024-05-27T08:41:15.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bc49e4700 1 Processor -- start 2024-05-27T08:41:15.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bc49e4700 1 -- start start 2024-05-27T08:41:15.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bc49e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bc0102470 0x7f5bc019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bc49e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bc0102e30 0x7f5bc019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bc49e4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc01084b0 0x7f5bc01a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bc49e4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc01125c0 con 0x7f5bc01084b0 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bc49e4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc0112440 con 0x7f5bc0102e30 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bc49e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc0112740 con 0x7f5bc0102470 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc01084b0 0x7f5bc01a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbed9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc01084b0 0x7f5bc01a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54500/0 (socket says 172.21.15.77:54500) 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbdd9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bc0102e30 0x7f5bc019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbe59c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bc0102470 0x7f5bc019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbed9d700 1 -- 172.21.15.77:0/3112264265 learned_addr learned my addr 172.21.15.77:0/3112264265 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:41:15.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbed9d700 1 -- 172.21.15.77:0/3112264265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bc0102470 msgr2=0x7f5bc019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbed9d700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bc0102470 0x7f5bc019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbed9d700 1 -- 172.21.15.77:0/3112264265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bc0102e30 msgr2=0x7f5bc019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbed9d700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bc0102e30 0x7f5bc019b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbed9d700 1 -- 172.21.15.77:0/3112264265 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc01a60f0 con 0x7f5bc01084b0 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.612+0000 7f5bbdd9b700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bc0102e30 0x7f5bc019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.613+0000 7f5bbed9d700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc01084b0 0x7f5bc01a5bb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800ada0 tx=0x7f5bb800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.613+0000 7f5baf7fe700 1 -- 172.21.15.77:0/3112264265 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8017020 con 0x7f5bc01084b0 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.613+0000 7f5baf7fe700 1 -- 172.21.15.77:0/3112264265 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bb800cd80 con 0x7f5bc01084b0 2024-05-27T08:41:15.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.613+0000 7f5baf7fe700 1 -- 172.21.15.77:0/3112264265 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8005260 con 0x7f5bc01084b0 2024-05-27T08:41:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.613+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc01a6320 con 0x7f5bc01084b0 2024-05-27T08:41:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.613+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc01a6880 con 0x7f5bc01084b0 2024-05-27T08:41:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.614+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc006a230 con 0x7f5bc01084b0 2024-05-27T08:41:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.615+0000 7f5baf7fe700 1 -- 172.21.15.77:0/3112264265 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5bb800a090 con 0x7f5bc01084b0 2024-05-27T08:41:15.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.616+0000 7f5baf7fe700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5ba80b2a70 0x7f5ba80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:15.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.616+0000 7f5bbe59c700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5ba80b2a70 0x7f5ba80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:15.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.616+0000 7f5baf7fe700 1 -- 172.21.15.77:0/3112264265 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5bb80e2280 con 0x7f5bc01084b0 2024-05-27T08:41:15.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.616+0000 7f5bbe59c700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5ba80b2a70 0x7f5ba80b4f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005950 tx=0x7f5bb000a300 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:41:15.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.619+0000 7f5baf7fe700 1 -- 172.21.15.77:0/3112264265 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5bb8021020 con 0x7f5bc01084b0 2024-05-27T08:41:15.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.876+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bc002e560 con 0x7f5bc01084b0 2024-05-27T08:41:15.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.877+0000 7f5baf7fe700 1 -- 172.21.15.77:0/3112264265 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11935 (secure 0 0 0) 0x7f5bb80a9d00 con 0x7f5bc01084b0 2024-05-27T08:41:15.878 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:41:15.878 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:40:48.425536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24433,24412,24410,24418,24434]},"id":1}]} 2024-05-27T08:41:15.880 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:15 smithi077 ceph-mon[32007]: pgmap v92: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-27T08:41:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5ba80b2a70 msgr2=0x7f5ba80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5ba80b2a70 0x7f5ba80b4f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005950 tx=0x7f5bb000a300 comp rx=0 tx=0).stop 2024-05-27T08:41:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc01084b0 msgr2=0x7f5bc01a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc01084b0 0x7f5bc01a5bb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800ada0 tx=0x7f5bb800c330 comp rx=0 tx=0).stop 2024-05-27T08:41:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 shutdown_connections 2024-05-27T08:41:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5ba80b2a70 0x7f5ba80b4f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5bc01084b0 0x7f5bc01a5bb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bc0102470 0x7f5bc019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 --2- 172.21.15.77:0/3112264265 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5bc0102e30 0x7f5bc019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.879+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 >> 172.21.15.77:0/3112264265 conn(0x7f5bc0079410 msgr2=0x7f5bc0100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:41:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.880+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 shutdown_connections 2024-05-27T08:41:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:15.880+0000 7f5bc49e4700 1 -- 172.21.15.77:0/3112264265 wait complete. 2024-05-27T08:41:15.882 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 32 2024-05-27T08:41:16.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:15 smithi179 ceph-mon[36481]: pgmap v92: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-27T08:41:16.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:15 smithi092 ceph-mon[36049]: pgmap v92: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-27T08:41:16.581 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 849bc6b8 --release --await' 2024-05-27T08:41:16.581 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db --set-id 849bc6b8 --release --await 2024-05-27T08:41:16.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.41 MB/sec warmup 70 sec latency 32722.615 ms 2024-05-27T08:41:16.832 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:41:16.859 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:16 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3112264265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:41:17.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:16 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3112264265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:41:17.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:16 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3112264265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:41:17.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.35 MB/sec warmup 71 sec latency 33722.642 ms 2024-05-27T08:41:18.065 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:17 smithi077 ceph-mon[32007]: pgmap v93: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-27T08:41:18.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:17 smithi179 ceph-mon[36481]: pgmap v93: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-27T08:41:18.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:17 smithi092 ceph-mon[36049]: pgmap v93: 97 pgs: 97 active+clean; 41 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-05-27T08:41:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 -- 172.21.15.77:0/1805799824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c06c470 msgr2=0x7fe17c1103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 --2- 172.21.15.77:0/1805799824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c06c470 0x7fe17c1103c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe17400b0a0 tx=0x7fe17400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 -- 172.21.15.77:0/1805799824 shutdown_connections 2024-05-27T08:41:18.552 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 --2- 172.21.15.77:0/1805799824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c06c470 0x7fe17c1103c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 --2- 172.21.15.77:0/1805799824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe17c1095a0 0x7fe17c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 --2- 172.21.15.77:0/1805799824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe17c06bab0 0x7fe17c06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 -- 172.21.15.77:0/1805799824 >> 172.21.15.77:0/1805799824 conn(0x7fe17c0793b0 msgr2=0x7fe17c0797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 -- 172.21.15.77:0/1805799824 shutdown_connections 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.549+0000 7fe181477700 1 -- 172.21.15.77:0/1805799824 wait complete. 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe181477700 1 Processor -- start 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe181477700 1 -- start start 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe181477700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe17c06bab0 0x7fe17c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe181477700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe17c06c470 0x7fe17c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe181477700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c1095a0 0x7fe17c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe181477700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c113750 con 0x7fe17c1095a0 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe181477700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c1135d0 con 0x7fe17c06bab0 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe181477700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c1138d0 con 0x7fe17c06c470 2024-05-27T08:41:18.553 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe17b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c1095a0 0x7fe17c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe17b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c1095a0 0x7fe17c199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54528/0 (socket says 172.21.15.77:54528) 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe17b7fe700 1 -- 172.21.15.77:0/2694619079 learned_addr learned my addr 172.21.15.77:0/2694619079 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.550+0000 7fe17affd700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe17c06bab0 0x7fe17c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe17b7fe700 1 -- 172.21.15.77:0/2694619079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe17c06c470 msgr2=0x7fe17c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe17c06c470 0x7fe17c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe17b7fe700 1 -- 172.21.15.77:0/2694619079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe17c06bab0 msgr2=0x7fe17c19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe17c06bab0 0x7fe17c19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe17b7fe700 1 -- 172.21.15.77:0/2694619079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17c19a1f0 con 0x7fe17c1095a0 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe17affd700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe17c06bab0 0x7fe17c19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c1095a0 0x7fe17c199a90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe17400b830 tx=0x7fe174004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe178ff9700 1 -- 172.21.15.77:0/2694619079 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174021070 con 0x7fe17c1095a0 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe178ff9700 1 -- 172.21.15.77:0/2694619079 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe174003c00 con 0x7fe17c1095a0 2024-05-27T08:41:18.554 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe178ff9700 1 -- 172.21.15.77:0/2694619079 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174004730 con 0x7fe17c1095a0 2024-05-27T08:41:18.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe181477700 1 -- 172.21.15.77:0/2694619079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17c19a480 con 0x7fe17c1095a0 2024-05-27T08:41:18.555 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.551+0000 7fe181477700 1 -- 172.21.15.77:0/2694619079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17c1a6750 con 0x7fe17c1095a0 2024-05-27T08:41:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.554+0000 7fe178ff9700 1 -- 172.21.15.77:0/2694619079 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe174007a00 con 0x7fe17c1095a0 2024-05-27T08:41:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.555+0000 7fe178ff9700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe1680b2a70 0x7fe1680b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.555+0000 7fe17affd700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe1680b2a70 0x7fe1680b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.555+0000 7fe178ff9700 1 -- 172.21.15.77:0/2694619079 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe17400f020 con 0x7fe17c1095a0 2024-05-27T08:41:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.555+0000 7fe18bd0db80 1 Processor -- start 2024-05-27T08:41:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.555+0000 7fe18bd0db80 1 -- start start 2024-05-27T08:41:18.556 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.555+0000 7fe17affd700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe1680b2a70 0x7fe1680b4f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe17c100b50 tx=0x7fe16c00b2c0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:41:18.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc42b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3dbf0 0x558a8fc3e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.560 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a8fb9be70 con 0x558a8fc3e5b0 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558a8fb8ba10 con 0x558a8fc42720 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558a8fb8fa50 con 0x558a8fc3dbf0 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54542/0 (socket says 172.21.15.77:54542) 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 -- 172.21.15.77:0/3076218238 learned_addr learned my addr 172.21.15.77:0/3076218238 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17affd700 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe173fff700 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3dbf0 0x558a8fc3e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 -- 172.21.15.77:0/3076218238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3dbf0 msgr2=0x558a8fc3e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3dbf0 0x558a8fc3e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 -- 172.21.15.77:0/3076218238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 msgr2=0x558a8fc42b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:41:18.561 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc42b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 -- 172.21.15.77:0/3076218238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8fb350b0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe173fff700 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3dbf0 0x558a8fc3e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4a620 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1740aaf80 tx=0x7fe174004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9608240408cada98 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe1717fa700 1 -- 172.21.15.77:0/3076218238 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1740aaff0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe1717fa700 1 -- 172.21.15.77:0/3076218238 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1740af3a0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.556+0000 7fe1717fa700 1 -- 172.21.15.77:0/3076218238 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1740ab470 con 0x558a8fc3e5b0 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 -- 172.21.15.77:0/3076218238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 msgr2=0x558a8fc4a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4a620 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1740aaf80 tx=0x7fe174004020 comp rx=0 tx=0).stop 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 -- 172.21.15.77:0/3076218238 shutdown_connections 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4a620 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc42b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/3076218238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3dbf0 0x558a8fc3e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.562 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 -- 172.21.15.77:0/3076218238 >> 172.21.15.77:0/3076218238 conn(0x558a8fb69340 msgr2=0x558a8fc388e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 -- 172.21.15.77:0/3076218238 shutdown_connections 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.557+0000 7fe18bd0db80 1 -- 172.21.15.77:0/3076218238 wait complete. 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 1 -- start start 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 1 Processor -- start 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc42720 0x558a8fc47b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558a8fa920a0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558a8faf3ea0 con 0x558a8fc3dbf0 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558a8fb9be70 con 0x558a8fc42720 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe18bd0db80 10 client.0 _ll_register_callbacks cb 0x558a8fb26a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:41:18.563 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe173fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe173fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54558/0 (socket says 172.21.15.77:54558) 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe173fff700 1 -- 172.21.15.77:0/740199655 learned_addr learned my addr 172.21.15.77:0/740199655 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe17affd700 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc42720 0x558a8fc47b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe173fff700 1 -- 172.21.15.77:0/740199655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc42720 msgr2=0x558a8fc47b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe173fff700 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc42720 0x558a8fc47b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe173fff700 1 -- 172.21.15.77:0/740199655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc3dbf0 msgr2=0x558a8fc4a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe173fff700 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.558+0000 7fe173fff700 1 -- 172.21.15.77:0/740199655 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8fadb9d0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe173fff700 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc475d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe16400c250 tx=0x7fe16400c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d94688e1e9433f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe1627fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:41:18.564 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe1627fc700 1 -- 172.21.15.77:0/740199655 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164015070 con 0x558a8fc3e5b0 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe1627fc700 1 -- 172.21.15.77:0/740199655 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe1640115f0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 1 -- 172.21.15.77:0/740199655 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1740af3a0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe1627fc700 1 -- 172.21.15.77:0/740199655 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe164021050 con 0x558a8fc3e5b0 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 10 client.15525 fetch_fsmap learned FSMap version 32 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 1 -- 172.21.15.77:0/740199655 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a8fb45860 con 0x558a8fc3e5b0 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe1627fc700 1 -- 172.21.15.77:0/740199655 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1740af3a0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe1627fc700 1 -- 172.21.15.77:0/740199655 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8374+0+0 (secure 0 0 0) 0x7fe1640198a0 con 0x558a8fc3e5b0 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 10 client.15525 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 10 client.15525 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 20 client.15525 populate_metadata read hostname 'smithi077' 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558a8fc574b0 0x558a8fc598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 4 client.15525 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.559+0000 7fe18bd0db80 1 -- 172.21.15.77:0/740199655 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a8fb9ddc0 con 0x558a8fc574b0 2024-05-27T08:41:18.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.560+0000 7fe17affd700 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558a8fc574b0 0x558a8fc598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.560+0000 7fe1627fc700 10 client.15525 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:41:18.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.560+0000 7fe17affd700 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558a8fc574b0 0x558a8fc598a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c85a0513faa211c8 server_cookie=764641b123ee51ba in_seq=0 out_seq=0 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe1627fc700 1 -- 172.21.15.77:0/740199655 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558a8fb9ddc0 con 0x558a8fc574b0 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe1627fc700 10 client.15525 handle_command_reply: tid=0 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 client.15525 shutdown 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 -- 172.21.15.77:0/740199655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 msgr2=0x558a8fc475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc475d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe16400c250 tx=0x7fe16400c560 comp rx=0 tx=0).stop 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 -- 172.21.15.77:0/740199655 shutdown_connections 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558a8fc574b0 0x558a8fc598a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc42720 0x558a8fc47b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe1627fc700 0 client.15525 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:41:18.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/740199655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3e5b0 0x558a8fc475d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 -- 172.21.15.77:0/740199655 >> 172.21.15.77:0/740199655 conn(0x558a8fb69340 msgr2=0x558a8fc3cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:41:18.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 -- 172.21.15.77:0/740199655 shutdown_connections 2024-05-27T08:41:18.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 1 -- 172.21.15.77:0/740199655 wait complete. 2024-05-27T08:41:18.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.561+0000 7fe18bd0db80 20 client.15525 trim_cache size 0 max 16384 2024-05-27T08:41:18.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe18bd0db80 1 Processor -- start 2024-05-27T08:41:18.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe18bd0db80 1 -- start start 2024-05-27T08:41:18.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc46530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc32e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc43ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c1addb0 con 0x558a8fc3dbf0 2024-05-27T08:41:18.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c1af9e0 con 0x558a8fc42720 2024-05-27T08:41:18.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c1ae480 con 0x558a8fc3e5b0 2024-05-27T08:41:18.583 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe17affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc46530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe17affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc46530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54574/0 (socket says 172.21.15.77:54574) 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.581+0000 7fe17affd700 1 -- 172.21.15.77:0/2108954033 learned_addr learned my addr 172.21.15.77:0/2108954033 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc43ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe173fff700 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc32e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe17affd700 1 -- 172.21.15.77:0/2108954033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 msgr2=0x558a8fc32e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe17affd700 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc32e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe17affd700 1 -- 172.21.15.77:0/2108954033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 msgr2=0x558a8fc43ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe17affd700 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc43ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe17affd700 1 -- 172.21.15.77:0/2108954033 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a8fc59f00 con 0x558a8fc3dbf0 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe173fff700 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc32e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe17affd700 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc46530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00bad0 tx=0x7fe16c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51fb14b12e567a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe170ff9700 1 -- 172.21.15.77:0/2108954033 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c020db0 con 0x558a8fc3dbf0 2024-05-27T08:41:18.584 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe170ff9700 1 -- 172.21.15.77:0/2108954033 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16c004ce0 con 0x558a8fc3dbf0 2024-05-27T08:41:18.585 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.582+0000 7fe170ff9700 1 -- 172.21.15.77:0/2108954033 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c029600 con 0x558a8fc3dbf0 2024-05-27T08:41:18.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 -- 172.21.15.77:0/2108954033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 msgr2=0x558a8fc46530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc46530 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00bad0 tx=0x7fe16c015000 comp rx=0 tx=0).stop 2024-05-27T08:41:18.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 -- 172.21.15.77:0/2108954033 shutdown_connections 2024-05-27T08:41:18.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc43ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc46530 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/2108954033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc32e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 -- 172.21.15.77:0/2108954033 >> 172.21.15.77:0/2108954033 conn(0x558a8fb69340 msgr2=0x558a8fc433c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 -- 172.21.15.77:0/2108954033 shutdown_connections 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 -- 172.21.15.77:0/2108954033 wait complete. 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.583+0000 7fe18bd0db80 1 -- start start 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe18bd0db80 1 Processor -- start 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe18bd0db80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc4b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c1af250 con 0x558a8fc3dbf0 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c1aeb50 con 0x558a8fc42720 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe18bd0db80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c1abd50 con 0x558a8fc3e5b0 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe18bd0db80 10 client.0 _ll_register_callbacks cb 0x558a8fb26a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:41:18.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17affd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54590/0 (socket says 172.21.15.77:54590) 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17affd700 1 -- 172.21.15.77:0/714390012 learned_addr learned my addr 172.21.15.77:0/714390012 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17b7fe700 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc4b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe173fff700 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17affd700 1 -- 172.21.15.77:0/714390012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 msgr2=0x558a8fc4b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17affd700 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17affd700 1 -- 172.21.15.77:0/714390012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 msgr2=0x558a8fc4b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17affd700 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc4b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.584+0000 7fe17affd700 1 -- 172.21.15.77:0/714390012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a8fc3ec30 con 0x558a8fc3dbf0 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe17affd700 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4abf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005cc0 tx=0x7fe16c020bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1426d8b3b45d66f server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe1627fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe1627fc700 1 -- 172.21.15.77:0/714390012 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c004950 con 0x558a8fc3dbf0 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe1627fc700 1 -- 172.21.15.77:0/714390012 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fe16c004e90 con 0x558a8fc3dbf0 2024-05-27T08:41:18.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 1 -- 172.21.15.77:0/714390012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe16c004ce0 con 0x558a8fc3dbf0 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe1627fc700 1 -- 172.21.15.77:0/714390012 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe16c004af0 con 0x558a8fc3dbf0 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 10 client.15531 fetch_fsmap learned FSMap version 32 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 1 -- 172.21.15.77:0/714390012 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a8fc3a200 con 0x558a8fc3dbf0 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe1627fc700 1 -- 172.21.15.77:0/714390012 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16c004ce0 con 0x558a8fc3dbf0 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe1627fc700 1 -- 172.21.15.77:0/714390012 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 32) ==== 8374+0+0 (secure 0 0 0) 0x7fe16c0293a0 con 0x558a8fc3dbf0 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 10 client.15531 fetch_fsmap finished waiting for FSMap version 32 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 10 client.15531 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 20 client.15531 populate_metadata read hostname 'smithi077' 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558a8fcc37e0 0x558a8fcc5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 4 client.15531 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "849bc6b8", "release": true, "await": true}] 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.585+0000 7fe18bd0db80 1 -- 172.21.15.77:0/714390012 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "set_id": "849bc6b8", "release": true, "await": true}) -- 0x558a8fb975a0 con 0x558a8fcc37e0 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.586+0000 7fe173fff700 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558a8fcc37e0 0x558a8fcc5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.586+0000 7fe1627fc700 10 client.15531 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:41:18.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.586+0000 7fe173fff700 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558a8fcc37e0 0x558a8fcc5c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a99f6f759e463c2 server_cookie=9037103ddd4020e in_seq=0 out_seq=0 2024-05-27T08:41:18.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 54149 4.29 MB/sec warmup 72 sec latency 34722.695 ms 2024-05-27T08:41:18.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe1627fc700 1 -- 172.21.15.77:0/714390012 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x558a8fb975a0 con 0x558a8fcc37e0 2024-05-27T08:41:18.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe1627fc700 10 client.15531 handle_command_reply: tid=0 2024-05-27T08:41:18.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 client.15531 shutdown 2024-05-27T08:41:18.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 -- 172.21.15.77:0/714390012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 msgr2=0x558a8fc4abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4abf0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005cc0 tx=0x7fe16c020bb0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 -- 172.21.15.77:0/714390012 shutdown_connections 2024-05-27T08:41:18.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558a8fcc37e0 0x558a8fcc5c00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558a8fc42720 0x558a8fc4b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558a8fc3dbf0 0x558a8fc4abf0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 --2- 172.21.15.77:0/714390012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558a8fc3e5b0 0x558a8fc4b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 -- 172.21.15.77:0/714390012 >> 172.21.15.77:0/714390012 conn(0x558a8fb69340 msgr2=0x558a8fc41e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe1627fc700 0 client.15531 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 -- 172.21.15.77:0/714390012 shutdown_connections 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 1 -- 172.21.15.77:0/714390012 wait complete. 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.594+0000 7fe18bd0db80 20 client.15531 trim_cache size 0 max 16384 2024-05-27T08:41:18.596 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 32, 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 8, 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "849bc6b8": { 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 8, 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:41:18.597 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 154.6, 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:41:18.598 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:41:18.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 -- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe1680b2a70 msgr2=0x7fe1680b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe1680b2a70 0x7fe1680b4f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe17c100b50 tx=0x7fe16c00b2c0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 -- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c1095a0 msgr2=0x7fe17c199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:41:18.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c1095a0 0x7fe17c199a90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe17400b830 tx=0x7fe174004d90 comp rx=0 tx=0).stop 2024-05-27T08:41:18.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 -- 172.21.15.77:0/2694619079 shutdown_connections 2024-05-27T08:41:18.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fe1680b2a70 0x7fe1680b4f30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe17c1095a0 0x7fe17c199a90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe17c06bab0 0x7fe17c19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 --2- 172.21.15.77:0/2694619079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe17c06c470 0x7fe17c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:41:18.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 -- 172.21.15.77:0/2694619079 >> 172.21.15.77:0/2694619079 conn(0x7fe17c0793b0 msgr2=0x7fe17c1010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:41:18.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 -- 172.21.15.77:0/2694619079 shutdown_connections 2024-05-27T08:41:18.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:41:18.595+0000 7fe181477700 1 -- 172.21.15.77:0/2694619079 wait complete. 2024-05-27T08:41:19.457 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '849bc6b8', total seconds elapsed: 40.4 2024-05-27T08:41:19.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 56726 4.45 MB/sec warmup 73 sec latency 34730.406 ms 2024-05-27T08:41:20.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:19 smithi179 ceph-mon[36481]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-05-27T08:41:20.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:19 smithi077 ceph-mon[32007]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-05-27T08:41:20.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:19 smithi092 ceph-mon[36049]: pgmap v94: 97 pgs: 97 active+clean; 40 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-05-27T08:41:20.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 57503 4.42 MB/sec warmup 74 sec latency 204.010 ms 2024-05-27T08:41:21.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 57503 4.36 MB/sec warmup 75 sec latency 1204.107 ms 2024-05-27T08:41:22.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:21 smithi179 ceph-mon[36481]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-05-27T08:41:22.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:21 smithi077 ceph-mon[32007]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-05-27T08:41:22.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:21 smithi092 ceph-mon[36049]: pgmap v95: 97 pgs: 97 active+clean; 40 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 29 KiB/s wr, 11 op/s 2024-05-27T08:41:22.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 57503 4.30 MB/sec warmup 76 sec latency 2204.191 ms 2024-05-27T08:41:23.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 58626 4.32 MB/sec warmup 77 sec latency 2712.375 ms 2024-05-27T08:41:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:23 smithi179 ceph-mon[36481]: pgmap v96: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 225 KiB/s wr, 57 op/s 2024-05-27T08:41:24.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:23 smithi077 ceph-mon[32007]: pgmap v96: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 225 KiB/s wr, 57 op/s 2024-05-27T08:41:24.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:23 smithi092 ceph-mon[36049]: pgmap v96: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 225 KiB/s wr, 57 op/s 2024-05-27T08:41:24.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 60715 4.44 MB/sec warmup 78 sec latency 10.754 ms 2024-05-27T08:41:25.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:24 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:41:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:24 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:41:25.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:24 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:41:25.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 61051 4.38 MB/sec warmup 79 sec latency 653.946 ms 2024-05-27T08:41:26.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:25 smithi179 ceph-mon[36481]: pgmap v97: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 199 KiB/s wr, 52 op/s 2024-05-27T08:41:26.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:25 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:41:26.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:25 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:41:26.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:25 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:41:26.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:25 smithi077 ceph-mon[32007]: pgmap v97: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 199 KiB/s wr, 52 op/s 2024-05-27T08:41:26.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:25 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:41:26.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:25 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:41:26.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:25 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:41:26.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:25 smithi092 ceph-mon[36049]: pgmap v97: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 199 KiB/s wr, 52 op/s 2024-05-27T08:41:26.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:25 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:41:26.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:25 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:41:26.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:25 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:41:26.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 61051 4.33 MB/sec warmup 80 sec latency 1654.026 ms 2024-05-27T08:41:27.591 INFO:tasks.workunit.client.0.smithi077.stdout: 1 61051 4.28 MB/sec warmup 81 sec latency 2654.085 ms 2024-05-27T08:41:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:27 smithi179 ceph-mon[36481]: pgmap v98: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 199 KiB/s wr, 52 op/s 2024-05-27T08:41:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:27 smithi077 ceph-mon[32007]: pgmap v98: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 199 KiB/s wr, 52 op/s 2024-05-27T08:41:28.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:27 smithi092 ceph-mon[36049]: pgmap v98: 97 pgs: 97 active+clean; 42 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 199 KiB/s wr, 52 op/s 2024-05-27T08:41:28.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 62394 4.29 MB/sec warmup 82 sec latency 3156.909 ms 2024-05-27T08:41:29.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 64289 4.40 MB/sec warmup 83 sec latency 10.331 ms 2024-05-27T08:41:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:29 smithi179 ceph-mon[36481]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 428 KiB/s wr, 95 op/s 2024-05-27T08:41:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:29 smithi077 ceph-mon[32007]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 428 KiB/s wr, 95 op/s 2024-05-27T08:41:30.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:29 smithi092 ceph-mon[36049]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 428 KiB/s wr, 95 op/s 2024-05-27T08:41:30.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 64660 4.35 MB/sec warmup 84 sec latency 682.416 ms 2024-05-27T08:41:31.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 64660 4.30 MB/sec warmup 85 sec latency 1682.514 ms 2024-05-27T08:41:32.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:31 smithi179 ceph-mon[36481]: pgmap v100: 97 pgs: 97 active+clean; 45 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 424 KiB/s wr, 88 op/s 2024-05-27T08:41:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:31 smithi077 ceph-mon[32007]: pgmap v100: 97 pgs: 97 active+clean; 45 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 424 KiB/s wr, 88 op/s 2024-05-27T08:41:32.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:31 smithi092 ceph-mon[36049]: pgmap v100: 97 pgs: 97 active+clean; 45 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 424 KiB/s wr, 88 op/s 2024-05-27T08:41:32.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 64660 4.25 MB/sec warmup 86 sec latency 2682.605 ms 2024-05-27T08:41:33.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 66070 4.26 MB/sec warmup 87 sec latency 3188.762 ms 2024-05-27T08:41:34.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:33 smithi179 ceph-mon[36481]: pgmap v101: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 739 KiB/s wr, 136 op/s 2024-05-27T08:41:34.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:33 smithi077 ceph-mon[32007]: pgmap v101: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 739 KiB/s wr, 136 op/s 2024-05-27T08:41:34.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:33 smithi092 ceph-mon[36049]: pgmap v101: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 739 KiB/s wr, 136 op/s 2024-05-27T08:41:34.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 67844 4.37 MB/sec warmup 88 sec latency 11.378 ms 2024-05-27T08:41:35.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 70028 4.41 MB/sec warmup 89 sec latency 12.663 ms 2024-05-27T08:41:36.116 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:35 smithi179 ceph-mon[36481]: pgmap v102: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 544 KiB/s wr, 90 op/s 2024-05-27T08:41:36.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:35 smithi077 ceph-mon[32007]: pgmap v102: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 544 KiB/s wr, 90 op/s 2024-05-27T08:41:36.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:35 smithi092 ceph-mon[36049]: pgmap v102: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 544 KiB/s wr, 90 op/s 2024-05-27T08:41:36.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 71624 4.48 MB/sec warmup 90 sec latency 14.678 ms 2024-05-27T08:41:37.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 74203 4.58 MB/sec warmup 91 sec latency 10.000 ms 2024-05-27T08:41:38.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:37 smithi179 ceph-mon[36481]: pgmap v103: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 544 KiB/s wr, 90 op/s 2024-05-27T08:41:38.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:37 smithi077 ceph-mon[32007]: pgmap v103: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 544 KiB/s wr, 90 op/s 2024-05-27T08:41:38.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:37 smithi092 ceph-mon[36049]: pgmap v103: 97 pgs: 97 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 544 KiB/s wr, 90 op/s 2024-05-27T08:41:38.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 75266 4.60 MB/sec warmup 92 sec latency 182.281 ms 2024-05-27T08:41:39.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 75266 4.55 MB/sec warmup 93 sec latency 1182.348 ms 2024-05-27T08:41:40.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:39 smithi179 ceph-mon[36481]: pgmap v104: 97 pgs: 97 active+clean; 53 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-05-27T08:41:40.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:39 smithi077 ceph-mon[32007]: pgmap v104: 97 pgs: 97 active+clean; 53 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-05-27T08:41:40.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:39 smithi092 ceph-mon[36049]: pgmap v104: 97 pgs: 97 active+clean; 53 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-05-27T08:41:40.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 75266 4.50 MB/sec warmup 94 sec latency 2182.442 ms 2024-05-27T08:41:41.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 75266 4.45 MB/sec warmup 95 sec latency 3182.489 ms 2024-05-27T08:41:42.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:41 smithi179 ceph-mon[36481]: pgmap v105: 97 pgs: 97 active+clean; 53 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 850 KiB/s wr, 142 op/s 2024-05-27T08:41:42.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:41 smithi077 ceph-mon[32007]: pgmap v105: 97 pgs: 97 active+clean; 53 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 850 KiB/s wr, 142 op/s 2024-05-27T08:41:42.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:41 smithi092 ceph-mon[36049]: pgmap v105: 97 pgs: 97 active+clean; 53 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 850 KiB/s wr, 142 op/s 2024-05-27T08:41:42.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 75266 4.40 MB/sec warmup 96 sec latency 4182.527 ms 2024-05-27T08:41:43.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 76437 4.41 MB/sec warmup 97 sec latency 4681.623 ms 2024-05-27T08:41:44.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:43 smithi179 ceph-mon[36481]: pgmap v106: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 983 KiB/s wr, 175 op/s 2024-05-27T08:41:44.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:43 smithi077 ceph-mon[32007]: pgmap v106: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 983 KiB/s wr, 175 op/s 2024-05-27T08:41:44.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:43 smithi092 ceph-mon[36049]: pgmap v106: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 983 KiB/s wr, 175 op/s 2024-05-27T08:41:44.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 78490 4.51 MB/sec warmup 98 sec latency 10.426 ms 2024-05-27T08:41:45.592 INFO:tasks.workunit.client.0.smithi077.stdout: 1 78877 4.47 MB/sec warmup 99 sec latency 661.258 ms 2024-05-27T08:41:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:45 smithi179 ceph-mon[36481]: pgmap v107: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 127 op/s 2024-05-27T08:41:46.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:45 smithi077 ceph-mon[32007]: pgmap v107: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 127 op/s 2024-05-27T08:41:46.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:45 smithi092 ceph-mon[36049]: pgmap v107: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 127 op/s 2024-05-27T08:41:46.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 78877 4.42 MB/sec warmup 100 sec latency 1661.359 ms 2024-05-27T08:41:47.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 78877 4.38 MB/sec warmup 101 sec latency 2661.461 ms 2024-05-27T08:41:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:47 smithi179 ceph-mon[36481]: pgmap v108: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 127 op/s 2024-05-27T08:41:48.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:47 smithi077 ceph-mon[32007]: pgmap v108: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 127 op/s 2024-05-27T08:41:48.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:47 smithi092 ceph-mon[36049]: pgmap v108: 97 pgs: 97 active+clean; 54 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 668 KiB/s wr, 127 op/s 2024-05-27T08:41:48.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 80264 4.39 MB/sec warmup 102 sec latency 3161.732 ms 2024-05-27T08:41:49.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 82045 4.47 MB/sec warmup 103 sec latency 14.693 ms 2024-05-27T08:41:50.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:49 smithi179 ceph-mon[36481]: pgmap v109: 97 pgs: 97 active+clean; 56 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 898 KiB/s wr, 173 op/s 2024-05-27T08:41:50.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:49 smithi077 ceph-mon[32007]: pgmap v109: 97 pgs: 97 active+clean; 56 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 898 KiB/s wr, 173 op/s 2024-05-27T08:41:50.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:49 smithi092 ceph-mon[36049]: pgmap v109: 97 pgs: 97 active+clean; 56 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 898 KiB/s wr, 173 op/s 2024-05-27T08:41:50.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 82358 4.43 MB/sec warmup 104 sec latency 700.556 ms 2024-05-27T08:41:51.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 82358 4.39 MB/sec warmup 105 sec latency 1700.666 ms 2024-05-27T08:41:52.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:51 smithi179 ceph-mon[36481]: pgmap v110: 97 pgs: 97 active+clean; 56 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 362 KiB/s wr, 78 op/s 2024-05-27T08:41:52.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:51 smithi077 ceph-mon[32007]: pgmap v110: 97 pgs: 97 active+clean; 56 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 362 KiB/s wr, 78 op/s 2024-05-27T08:41:52.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:51 smithi092 ceph-mon[36049]: pgmap v110: 97 pgs: 97 active+clean; 56 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 362 KiB/s wr, 78 op/s 2024-05-27T08:41:52.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 82358 4.35 MB/sec warmup 106 sec latency 2700.711 ms 2024-05-27T08:41:53.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 83777 4.36 MB/sec warmup 107 sec latency 3198.359 ms 2024-05-27T08:41:54.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:53 smithi179 ceph-mon[36481]: pgmap v111: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 723 KiB/s wr, 129 op/s 2024-05-27T08:41:54.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:53 smithi077 ceph-mon[32007]: pgmap v111: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 723 KiB/s wr, 129 op/s 2024-05-27T08:41:54.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:53 smithi092 ceph-mon[36049]: pgmap v111: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 723 KiB/s wr, 129 op/s 2024-05-27T08:41:54.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 85579 4.44 MB/sec warmup 108 sec latency 23.916 ms 2024-05-27T08:41:55.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 85843 4.40 MB/sec warmup 109 sec latency 717.013 ms 2024-05-27T08:41:56.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:55 smithi179 ceph-mon[36481]: pgmap v112: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 590 KiB/s wr, 97 op/s 2024-05-27T08:41:56.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:55 smithi077 ceph-mon[32007]: pgmap v112: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 590 KiB/s wr, 97 op/s 2024-05-27T08:41:56.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:55 smithi092 ceph-mon[36049]: pgmap v112: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 590 KiB/s wr, 97 op/s 2024-05-27T08:41:56.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 85843 4.36 MB/sec warmup 110 sec latency 1717.110 ms 2024-05-27T08:41:57.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 85843 4.32 MB/sec warmup 111 sec latency 2717.199 ms 2024-05-27T08:41:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:57 smithi179 ceph-mon[36481]: pgmap v113: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 590 KiB/s wr, 97 op/s 2024-05-27T08:41:58.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:57 smithi077 ceph-mon[32007]: pgmap v113: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 590 KiB/s wr, 97 op/s 2024-05-27T08:41:58.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:57 smithi092 ceph-mon[36049]: pgmap v113: 97 pgs: 97 active+clean; 57 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 590 KiB/s wr, 97 op/s 2024-05-27T08:41:58.593 INFO:tasks.workunit.client.0.smithi077.stdout: 1 86952 4.33 MB/sec warmup 112 sec latency 3217.217 ms 2024-05-27T08:41:59.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89081 4.42 MB/sec warmup 113 sec latency 16.860 ms 2024-05-27T08:42:00.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:41:59 smithi179 ceph-mon[36481]: pgmap v114: 97 pgs: 97 active+clean; 59 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-27T08:42:00.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:41:59 smithi077 ceph-mon[32007]: pgmap v114: 97 pgs: 97 active+clean; 59 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-27T08:42:00.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:41:59 smithi092 ceph-mon[36049]: pgmap v114: 97 pgs: 97 active+clean; 59 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-05-27T08:42:00.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89392 4.38 MB/sec warmup 114 sec latency 638.732 ms 2024-05-27T08:42:01.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89448 4.34 MB/sec warmup 115 sec latency 1021.440 ms 2024-05-27T08:42:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:01 smithi179 ceph-mon[36481]: pgmap v115: 97 pgs: 97 active+clean; 59 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 982 KiB/s wr, 108 op/s 2024-05-27T08:42:02.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:01 smithi077 ceph-mon[32007]: pgmap v115: 97 pgs: 97 active+clean; 59 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 982 KiB/s wr, 108 op/s 2024-05-27T08:42:02.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:01 smithi092 ceph-mon[36049]: pgmap v115: 97 pgs: 97 active+clean; 59 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 919 KiB/s rd, 982 KiB/s wr, 108 op/s 2024-05-27T08:42:02.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89448 4.30 MB/sec warmup 116 sec latency 1581.868 ms 2024-05-27T08:42:03.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89516 4.27 MB/sec warmup 117 sec latency 2079.524 ms 2024-05-27T08:42:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:03 smithi179 ceph-mon[36481]: pgmap v116: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-27T08:42:04.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:03 smithi077 ceph-mon[32007]: pgmap v116: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-27T08:42:04.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:03 smithi092 ceph-mon[36049]: pgmap v116: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-05-27T08:42:04.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89516 4.23 MB/sec warmup 118 sec latency 1451.811 ms 2024-05-27T08:42:05.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89516 4.20 MB/sec warmup 119 sec latency 2451.897 ms 2024-05-27T08:42:06.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:05 smithi179 ceph-mon[36481]: pgmap v117: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-27T08:42:06.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:05 smithi077 ceph-mon[32007]: pgmap v117: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-27T08:42:06.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:05 smithi092 ceph-mon[36049]: pgmap v117: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-27T08:42:07.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89516 0.00 MB/sec execute 1 sec latency 4452.062 ms 2024-05-27T08:42:08.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:07 smithi179 ceph-mon[36481]: pgmap v118: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-27T08:42:08.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:07 smithi077 ceph-mon[32007]: pgmap v118: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-27T08:42:08.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:07 smithi092 ceph-mon[36049]: pgmap v118: 97 pgs: 97 active+clean; 61 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-05-27T08:42:08.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 89882 0.82 MB/sec execute 2 sec latency 4957.005 ms 2024-05-27T08:42:09.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 92491 6.14 MB/sec execute 3 sec latency 10.808 ms 2024-05-27T08:42:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:09 smithi179 ceph-mon[36481]: pgmap v119: 97 pgs: 97 active+clean; 63 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-27T08:42:10.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:09 smithi077 ceph-mon[32007]: pgmap v119: 97 pgs: 97 active+clean; 63 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-27T08:42:10.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:09 smithi092 ceph-mon[36049]: pgmap v119: 97 pgs: 97 active+clean; 63 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-05-27T08:42:10.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 92992 4.74 MB/sec execute 4 sec latency 386.797 ms 2024-05-27T08:42:11.594 INFO:tasks.workunit.client.0.smithi077.stdout: 1 92992 3.79 MB/sec execute 5 sec latency 1386.915 ms 2024-05-27T08:42:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:11 smithi179 ceph-mon[36481]: pgmap v120: 97 pgs: 97 active+clean; 63 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 799 KiB/s wr, 88 op/s 2024-05-27T08:42:12.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:11 smithi077 ceph-mon[32007]: pgmap v120: 97 pgs: 97 active+clean; 63 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 799 KiB/s wr, 88 op/s 2024-05-27T08:42:12.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:11 smithi092 ceph-mon[36049]: pgmap v120: 97 pgs: 97 active+clean; 63 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 799 KiB/s wr, 88 op/s 2024-05-27T08:42:12.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 92992 3.16 MB/sec execute 6 sec latency 2387.040 ms 2024-05-27T08:42:13.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 94074 3.44 MB/sec execute 7 sec latency 2887.293 ms 2024-05-27T08:42:14.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:13 smithi179 ceph-mon[36481]: pgmap v121: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-05-27T08:42:14.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:13 smithi077 ceph-mon[32007]: pgmap v121: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-05-27T08:42:14.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:13 smithi092 ceph-mon[36049]: pgmap v121: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-05-27T08:42:14.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 96206 4.73 MB/sec execute 8 sec latency 11.941 ms 2024-05-27T08:42:15.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 96483 4.22 MB/sec execute 9 sec latency 685.173 ms 2024-05-27T08:42:16.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:15 smithi179 ceph-mon[36481]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-27T08:42:16.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:15 smithi077 ceph-mon[32007]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-27T08:42:16.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:15 smithi092 ceph-mon[36049]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-27T08:42:16.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 96483 3.79 MB/sec execute 10 sec latency 1685.264 ms 2024-05-27T08:42:17.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 96483 3.45 MB/sec execute 11 sec latency 2685.363 ms 2024-05-27T08:42:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:17 smithi179 ceph-mon[36481]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-27T08:42:18.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:17 smithi077 ceph-mon[32007]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-27T08:42:18.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:17 smithi092 ceph-mon[36049]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 101 op/s 2024-05-27T08:42:18.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 97708 3.60 MB/sec execute 12 sec latency 3183.846 ms 2024-05-27T08:42:19.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 99753 4.38 MB/sec execute 13 sec latency 10.320 ms 2024-05-27T08:42:20.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:19 smithi179 ceph-mon[36481]: pgmap v124: 97 pgs: 97 active+clean; 64 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-05-27T08:42:20.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:19 smithi077 ceph-mon[32007]: pgmap v124: 97 pgs: 97 active+clean; 64 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-05-27T08:42:20.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:19 smithi092 ceph-mon[36049]: pgmap v124: 97 pgs: 97 active+clean; 64 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-05-27T08:42:20.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 100028 4.08 MB/sec execute 14 sec latency 715.403 ms 2024-05-27T08:42:21.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 100028 3.81 MB/sec execute 15 sec latency 1715.451 ms 2024-05-27T08:42:22.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:21 smithi179 ceph-mon[36481]: pgmap v125: 97 pgs: 97 active+clean; 64 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-27T08:42:22.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:21 smithi077 ceph-mon[32007]: pgmap v125: 97 pgs: 97 active+clean; 64 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-27T08:42:22.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:21 smithi092 ceph-mon[36049]: pgmap v125: 97 pgs: 97 active+clean; 64 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-05-27T08:42:22.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 100028 3.57 MB/sec execute 16 sec latency 2715.551 ms 2024-05-27T08:42:23.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 100920 3.65 MB/sec execute 17 sec latency 3218.117 ms 2024-05-27T08:42:23.715 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-05-27T08:42:23.716 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:42:23.976 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:42:24.101 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:23 smithi077 ceph-mon[32007]: pgmap v126: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-05-27T08:42:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:23 smithi179 ceph-mon[36481]: pgmap v126: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-05-27T08:42:24.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:23 smithi092 ceph-mon[36049]: pgmap v126: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-05-27T08:42:24.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103196 4.21 MB/sec execute 18 sec latency 9.923 ms 2024-05-27T08:42:25.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:24 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:42:25.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:24 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:42:25.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:24 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:42:25.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103632 4.02 MB/sec execute 19 sec latency 555.157 ms 2024-05-27T08:42:25.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 -- 172.21.15.77:0/1252622488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 msgr2=0x7f15d406b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:25.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 --2- 172.21.15.77:0/1252622488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 0x7f15d406b6a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009990 tx=0x7f15c8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 -- 172.21.15.77:0/1252622488 shutdown_connections 2024-05-27T08:42:25.700 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 --2- 172.21.15.77:0/1252622488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15d406bbe0 0x7f15d41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 --2- 172.21.15.77:0/1252622488 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d41095b0 0x7f15d4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 --2- 172.21.15.77:0/1252622488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 0x7f15d406b6a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 -- 172.21.15.77:0/1252622488 >> 172.21.15.77:0/1252622488 conn(0x7f15d40793f0 msgr2=0x7f15d4079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 -- 172.21.15.77:0/1252622488 shutdown_connections 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.695+0000 7f15d8818700 1 -- 172.21.15.77:0/1252622488 wait complete. 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.696+0000 7f15d8818700 1 Processor -- start 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.696+0000 7f15d8818700 1 -- start start 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.696+0000 7f15d8818700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 0x7f15d419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.696+0000 7f15d8818700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d406bbe0 0x7f15d419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:25.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.696+0000 7f15d8818700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15d41095b0 0x7f15d41a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:25.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d8818700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41136e0 con 0x7f15d406b220 2024-05-27T08:42:25.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d8818700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4113560 con 0x7f15d406bbe0 2024-05-27T08:42:25.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d8818700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4113860 con 0x7f15d41095b0 2024-05-27T08:42:25.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 0x7f15d419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:25.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 0x7f15d419be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60530/0 (socket says 172.21.15.77:60530) 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 -- 172.21.15.77:0/740338312 learned_addr learned my addr 172.21.15.77:0/740338312 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d1d9b700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d406bbe0 0x7f15d419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d2d9d700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15d41095b0 0x7f15d41a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 -- 172.21.15.77:0/740338312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15d41095b0 msgr2=0x7f15d41a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15d41095b0 0x7f15d41a0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 -- 172.21.15.77:0/740338312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d406bbe0 msgr2=0x7f15d419c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d406bbe0 0x7f15d419c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 -- 172.21.15.77:0/740338312 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d41a0e60 con 0x7f15d406b220 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d2d9d700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15d41095b0 0x7f15d41a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d259c700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 0x7f15d419be50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f15c400e9a0 tx=0x7f15c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:25.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15c37fe700 1 -- 172.21.15.77:0/740338312 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c400cbd0 con 0x7f15d406b220 2024-05-27T08:42:25.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41a1150 con 0x7f15d406b220 2024-05-27T08:42:25.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15c37fe700 1 -- 172.21.15.77:0/740338312 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15c400cd70 con 0x7f15d406b220 2024-05-27T08:42:25.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15c37fe700 1 -- 172.21.15.77:0/740338312 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4018680 con 0x7f15d406b220 2024-05-27T08:42:25.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.697+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41a89d0 con 0x7f15d406b220 2024-05-27T08:42:25.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.699+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d406a230 con 0x7f15d406b220 2024-05-27T08:42:25.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.700+0000 7f15c37fe700 1 -- 172.21.15.77:0/740338312 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f15c4010430 con 0x7f15d406b220 2024-05-27T08:42:25.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.700+0000 7f15c37fe700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f15bc0b29f0 0x7f15bc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:25.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.701+0000 7f15d1d9b700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f15bc0b29f0 0x7f15bc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:25.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.701+0000 7f15c37fe700 1 -- 172.21.15.77:0/740338312 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f15c40e2840 con 0x7f15d406b220 2024-05-27T08:42:25.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.701+0000 7f15d1d9b700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f15bc0b29f0 0x7f15bc0b4eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15c800b2e0 tx=0x7f15c8005960 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:25.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.705+0000 7f15c37fe700 1 -- 172.21.15.77:0/740338312 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15c40aa230 con 0x7f15d406b220 2024-05-27T08:42:25.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.970+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15d419d1b0 con 0x7f15d406b220 2024-05-27T08:42:25.971 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:25 smithi077 ceph-mon[32007]: pgmap v127: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-05-27T08:42:25.971 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:25 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:42:25.971 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.970+0000 7f15c37fe700 1 -- 172.21.15.77:0/740338312 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11934 (secure 0 0 0) 0x7f15c40b22b0 con 0x7f15d406b220 2024-05-27T08:42:25.972 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:42:25.972 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:42:23.427101+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f15bc0b29f0 msgr2=0x7f15bc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f15bc0b29f0 0x7f15bc0b4eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15c800b2e0 tx=0x7f15c8005960 comp rx=0 tx=0).stop 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 msgr2=0x7f15d419be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 0x7f15d419be50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f15c400e9a0 tx=0x7f15c400ecb0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 shutdown_connections 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f15bc0b29f0 0x7f15bc0b4eb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15d41095b0 0x7f15d41a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f15d406b220 0x7f15d419be50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 --2- 172.21.15.77:0/740338312 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f15d406bbe0 0x7f15d419c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 >> 172.21.15.77:0/740338312 conn(0x7f15d40793f0 msgr2=0x7f15d410eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:25.974 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 shutdown_connections 2024-05-27T08:42:25.975 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:25.973+0000 7f15d8818700 1 -- 172.21.15.77:0/740338312 wait complete. 2024-05-27T08:42:25.975 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 33 2024-05-27T08:42:26.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:25 smithi092 ceph-mon[36049]: pgmap v127: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-05-27T08:42:26.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:25 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:42:26.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:25 smithi179 ceph-mon[36481]: pgmap v127: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-05-27T08:42:26.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:25 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:42:26.595 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103632 3.82 MB/sec execute 20 sec latency 1555.240 ms 2024-05-27T08:42:26.744 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-05-27T08:42:26.744 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-05-27T08:42:27.002 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:42:27.031 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:26 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:42:27.031 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:26 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:42:27.031 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:26 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:42:27.031 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:26 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/740338312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:42:27.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:26 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:42:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:26 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:42:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:26 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:42:27.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:26 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/740338312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:42:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:26 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:42:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:26 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:42:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:26 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:42:27.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:26 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/740338312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:42:27.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103632 3.64 MB/sec execute 21 sec latency 2555.269 ms 2024-05-27T08:42:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:27 smithi077 ceph-mon[32007]: pgmap v128: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-05-27T08:42:28.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:27 smithi092 ceph-mon[36049]: pgmap v128: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-05-27T08:42:28.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:27 smithi179 ceph-mon[36481]: pgmap v128: 97 pgs: 97 active+clean; 69 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 1.0 MiB/s wr, 104 op/s 2024-05-27T08:42:28.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103700 3.48 MB/sec execute 22 sec latency 3056.523 ms 2024-05-27T08:42:28.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 -- 172.21.15.77:0/1728176667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff490102460 msgr2=0x7ff4901028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1728176667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff490102460 0x7ff4901028e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff478009990 tx=0x7ff478009ca0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.659 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 -- 172.21.15.77:0/1728176667 shutdown_connections 2024-05-27T08:42:28.660 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1728176667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490102e20 0x7ff49010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1728176667 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff4901073c0 0x7ff4901077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1728176667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff490102460 0x7ff4901028e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 -- 172.21.15.77:0/1728176667 >> 172.21.15.77:0/1728176667 conn(0x7ff490079490 msgr2=0x7ff4900798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 -- 172.21.15.77:0/1728176667 shutdown_connections 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 -- 172.21.15.77:0/1728176667 wait complete. 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 Processor -- start 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.657+0000 7ff4965c1700 1 -- start start 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff4965c1700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff490102460 0x7ff4901988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff4965c1700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490102e20 0x7ff490198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff4965c1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4901073c0 0x7ff490199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff4965c1700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901126f0 con 0x7ff4901073c0 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff4965c1700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff490112570 con 0x7ff490102460 2024-05-27T08:42:28.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff4965c1700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff490112870 con 0x7ff490102e20 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4901073c0 0x7ff490199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4901073c0 0x7ff490199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60548/0 (socket says 172.21.15.77:60548) 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 -- 172.21.15.77:0/1599535410 learned_addr learned my addr 172.21.15.77:0/1599535410 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff48ffff700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff490102460 0x7ff4901988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff487fff700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490102e20 0x7ff490198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 -- 172.21.15.77:0/1599535410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490102e20 msgr2=0x7ff490198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490102e20 0x7ff490198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 -- 172.21.15.77:0/1599535410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff490102460 msgr2=0x7ff4901988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff490102460 0x7ff4901988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 -- 172.21.15.77:0/1599535410 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901a8150 con 0x7ff4901073c0 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.658+0000 7ff494b5e700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4901073c0 0x7ff490199360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b970 tx=0x7ff48800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.659+0000 7ff48dffb700 1 -- 172.21.15.77:0/1599535410 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48800d860 con 0x7ff4901073c0 2024-05-27T08:42:28.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.659+0000 7ff4965c1700 1 -- 172.21.15.77:0/1599535410 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4901a8440 con 0x7ff4901073c0 2024-05-27T08:42:28.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.659+0000 7ff48dffb700 1 -- 172.21.15.77:0/1599535410 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48800e430 con 0x7ff4901073c0 2024-05-27T08:42:28.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.659+0000 7ff48dffb700 1 -- 172.21.15.77:0/1599535410 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488017660 con 0x7ff4901073c0 2024-05-27T08:42:28.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.659+0000 7ff4965c1700 1 -- 172.21.15.77:0/1599535410 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901a8950 con 0x7ff4901073c0 2024-05-27T08:42:28.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.661+0000 7ff48dffb700 1 -- 172.21.15.77:0/1599535410 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff48800da00 con 0x7ff4901073c0 2024-05-27T08:42:28.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff48dffb700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff47c0b2aa0 0x7ff47c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff48ffff700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff47c0b2aa0 0x7ff47c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff48dffb700 1 -- 172.21.15.77:0/1599535410 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff4880e21f0 con 0x7ff4901073c0 2024-05-27T08:42:28.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff48ffff700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff47c0b2aa0 0x7ff47c0b4f60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff480009a10 tx=0x7ff480009190 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:28.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff4a0e57b80 1 Processor -- start 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff4a0e57b80 1 -- start start 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c806057b0 0x558c80605b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80600c80 0x558c80601100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80601640 0x558c8060d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c8055ef00 con 0x558c806057b0 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558c8054eaa0 con 0x558c80601640 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.662+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558c80552ae0 con 0x558c80600c80 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c806057b0 0x558c80605b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c806057b0 0x558c80605b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60560/0 (socket says 172.21.15.77:60560) 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 -- 172.21.15.77:0/2133655896 learned_addr learned my addr 172.21.15.77:0/2133655896 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff487fff700 1 --2- 172.21.15.77:0/2133655896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80600c80 0x558c80601100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 -- 172.21.15.77:0/2133655896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80600c80 msgr2=0x558c80601100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 --2- 172.21.15.77:0/2133655896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80600c80 0x558c80601100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 -- 172.21.15.77:0/2133655896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80601640 msgr2=0x558c8060d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 --2- 172.21.15.77:0/2133655896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80601640 0x558c8060d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 -- 172.21.15.77:0/2133655896 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c804f8140 con 0x558c806057b0 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff48ffff700 1 --2- 172.21.15.77:0/2133655896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c806057b0 0x558c80605b90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff48000ea00 tx=0x7ff480011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd9863704909b76 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff4867fc700 1 -- 172.21.15.77:0/2133655896 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48001b040 con 0x558c806057b0 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff4867fc700 1 -- 172.21.15.77:0/2133655896 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff480024070 con 0x558c806057b0 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.663+0000 7ff4867fc700 1 -- 172.21.15.77:0/2133655896 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480013740 con 0x558c806057b0 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/2133655896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c806057b0 msgr2=0x558c80605b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/2133655896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c806057b0 0x558c80605b90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff48000ea00 tx=0x7ff480011000 comp rx=0 tx=0).stop 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/2133655896 shutdown_connections 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/2133655896 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80601640 0x558c8060d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/2133655896 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c806057b0 0x558c80605b90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/2133655896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80600c80 0x558c80601100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.670 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/2133655896 >> 172.21.15.77:0/2133655896 conn(0x558c8052c3d0 msgr2=0x558c805fb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/2133655896 shutdown_connections 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/2133655896 wait complete. 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.664+0000 7ff4a0e57b80 1 -- start start 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff4a0e57b80 1 Processor -- start 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80600c80 0x558c8060d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80601640 0x558c8060a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c806057b0 0x558c8060ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558c80455130 con 0x558c80600c80 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x558c804b6f30 con 0x558c806057b0 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558c8060dd10 con 0x558c80601640 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff4a0e57b80 10 client.0 _ll_register_callbacks cb 0x558c804e9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff48ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80600c80 0x558c8060d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.671 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff48ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80600c80 0x558c8060d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60570/0 (socket says 172.21.15.77:60570) 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff48ffff700 1 -- 172.21.15.77:0/1038515236 learned_addr learned my addr 172.21.15.77:0/1038515236 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff487fff700 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80601640 0x558c8060a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff494b5e700 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c806057b0 0x558c8060ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff48ffff700 1 -- 172.21.15.77:0/1038515236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80601640 msgr2=0x558c8060a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff48ffff700 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80601640 0x558c8060a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff48ffff700 1 -- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c806057b0 msgr2=0x558c8060ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.665+0000 7ff48ffff700 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c806057b0 0x558c8060ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff48ffff700 1 -- 172.21.15.77:0/1038515236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8049ea60 con 0x558c80600c80 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff494b5e700 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c806057b0 0x558c8060ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff48ffff700 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80600c80 0x558c8060d340 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff480015a00 tx=0x7ff480015a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62938b4bfeefd554 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff46f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff46f7fe700 1 -- 172.21.15.77:0/1038515236 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48002d070 con 0x558c80600c80 2024-05-27T08:42:28.672 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff46f7fe700 1 -- 172.21.15.77:0/1038515236 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff480029810 con 0x558c80600c80 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/1038515236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff480024070 con 0x558c80600c80 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff46f7fe700 1 -- 172.21.15.77:0/1038515236 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff480029a10 con 0x558c80600c80 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff4a0e57b80 10 client.15549 fetch_fsmap learned FSMap version 33 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/1038515236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558c805088f0 con 0x558c80600c80 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff46f7fe700 1 -- 172.21.15.77:0/1038515236 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff480024070 con 0x558c80600c80 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff46f7fe700 1 -- 172.21.15.77:0/1038515236 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff48001e030 con 0x558c80600c80 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff4a0e57b80 10 client.15549 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff4a0e57b80 10 client.15549 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.666+0000 7ff4a0e57b80 20 client.15549 populate_metadata read hostname 'smithi077' 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.667+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558c8061a770 0x558c8061cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.667+0000 7ff4a0e57b80 4 client.15549 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.667+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/1038515236 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c80560e50 con 0x558c8061a770 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.667+0000 7ff487fff700 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558c8061a770 0x558c8061cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.667+0000 7ff46f7fe700 10 client.15549 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:42:28.673 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.667+0000 7ff487fff700 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558c8061a770 0x558c8061cb60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81eb5203f90011f7 server_cookie=34e0d6a9d6d7c4ae in_seq=0 out_seq=0 2024-05-27T08:42:28.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff46f7fe700 1 -- 172.21.15.77:0/1038515236 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558c80560e50 con 0x558c8061a770 2024-05-27T08:42:28.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff46f7fe700 10 client.15549 handle_command_reply: tid=0 2024-05-27T08:42:28.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 client.15549 shutdown 2024-05-27T08:42:28.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/1038515236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80600c80 msgr2=0x558c8060d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80600c80 0x558c8060d340 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff480015a00 tx=0x7ff480015a30 comp rx=0 tx=0).stop 2024-05-27T08:42:28.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/1038515236 shutdown_connections 2024-05-27T08:42:28.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558c8061a770 0x558c8061cb60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c806057b0 0x558c8060ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80600c80 0x558c8060d340 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/1038515236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c80601640 0x558c8060a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/1038515236 >> 172.21.15.77:0/1038515236 conn(0x558c8052c3d0 msgr2=0x558c805ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:28.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff46f7fe700 0 client.15549 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:42:28.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/1038515236 shutdown_connections 2024-05-27T08:42:28.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/1038515236 wait complete. 2024-05-27T08:42:28.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.669+0000 7ff4a0e57b80 20 client.15549 trim_cache size 0 max 16384 2024-05-27T08:42:28.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff4a0e57b80 1 Processor -- start 2024-05-27T08:42:28.689 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff4a0e57b80 1 -- start start 2024-05-27T08:42:28.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c805f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c80608e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 0x558c80609340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b0060 con 0x558c80601640 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b1c60 con 0x558c80600c80 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b0730 con 0x558c806057b0 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c80608e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c80608e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60586/0 (socket says 172.21.15.77:60586) 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 -- 172.21.15.77:0/505549880 learned_addr learned my addr 172.21.15.77:0/505549880 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff48ffff700 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c805f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 -- 172.21.15.77:0/505549880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 msgr2=0x558c80609340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 0x558c80609340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 -- 172.21.15.77:0/505549880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 msgr2=0x558c805f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c805f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 -- 172.21.15.77:0/505549880 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c80608bb0 con 0x558c80601640 2024-05-27T08:42:28.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff48ffff700 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c805f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.689+0000 7ff487fff700 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c80608e00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff478009750 tx=0x7ff478005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cd0a5d859630dc server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff485ffb700 1 -- 172.21.15.77:0/505549880 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47800be90 con 0x558c80601640 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff485ffb700 1 -- 172.21.15.77:0/505549880 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff478003d10 con 0x558c80601640 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff485ffb700 1 -- 172.21.15.77:0/505549880 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47800f6c0 con 0x558c80601640 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/505549880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 msgr2=0x558c80608e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c80608e00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff478009750 tx=0x7ff478005960 comp rx=0 tx=0).stop 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/505549880 shutdown_connections 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 0x558c80609340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c805f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/505549880 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c80608e00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/505549880 >> 172.21.15.77:0/505549880 conn(0x558c8052c3d0 msgr2=0x558c8060c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/505549880 shutdown_connections 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.690+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/505549880 wait complete. 2024-05-27T08:42:28.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 1 -- start start 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 1 Processor -- start 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c80612f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c806134a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 0x558c8060ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b14d0 con 0x558c80601640 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b0e00 con 0x558c80600c80 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901ae000 con 0x558c806057b0 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff4a0e57b80 10 client.0 _ll_register_callbacks cb 0x558c804e9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff487fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c806134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff487fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c806134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60594/0 (socket says 172.21.15.77:60594) 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff487fff700 1 -- 172.21.15.77:0/874041239 learned_addr learned my addr 172.21.15.77:0/874041239 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff48ffff700 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c80612f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.704 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.691+0000 7ff494b5e700 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 0x558c8060ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff487fff700 1 -- 172.21.15.77:0/874041239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 msgr2=0x558c8060ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff487fff700 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 0x558c8060ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff487fff700 1 -- 172.21.15.77:0/874041239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 msgr2=0x558c80612f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff487fff700 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c80612f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff487fff700 1 -- 172.21.15.77:0/874041239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c80610470 con 0x558c80601640 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff487fff700 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c806134a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff478009ce0 tx=0x7ff478004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80732e8e9220affe server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff46f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff46f7fe700 1 -- 172.21.15.77:0/874041239 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47803e060 con 0x558c80601640 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff46f7fe700 1 -- 172.21.15.77:0/874041239 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff47800bc90 con 0x558c80601640 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/874041239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff478003d10 con 0x558c80601640 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff46f7fe700 1 -- 172.21.15.77:0/874041239 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff4780050d0 con 0x558c80601640 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff4a0e57b80 10 client.15555 fetch_fsmap learned FSMap version 33 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/874041239 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x558c80685910 con 0x558c80601640 2024-05-27T08:42:28.705 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff46f7fe700 1 -- 172.21.15.77:0/874041239 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff478003d10 con 0x558c80601640 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff46f7fe700 1 -- 172.21.15.77:0/874041239 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7ff478030830 con 0x558c80601640 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff4a0e57b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff4a0e57b80 10 client.15555 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.692+0000 7ff4a0e57b80 20 client.15555 populate_metadata read hostname 'smithi077' 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.693+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558c80685ff0 0x558c806883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.693+0000 7ff4a0e57b80 4 client.15555 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.693+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/874041239 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x558c8055a630 con 0x558c80685ff0 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.693+0000 7ff48ffff700 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558c80685ff0 0x558c806883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.693+0000 7ff48ffff700 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558c80685ff0 0x558c806883e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17523c3452949e40 server_cookie=9f025cc92c2c5d65 in_seq=0 out_seq=0 2024-05-27T08:42:28.706 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.693+0000 7ff46f7fe700 10 client.15555 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:42:28.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff46f7fe700 1 -- 172.21.15.77:0/874041239 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558c8055a630 con 0x558c80685ff0 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff46f7fe700 10 client.15555 handle_command_reply: tid=0 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff4a0e57b80 1 client.15555 shutdown 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/874041239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 msgr2=0x558c806134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c806134a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff478009ce0 tx=0x7ff478004f40 comp rx=0 tx=0).stop 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/874041239 shutdown_connections 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x558c80685ff0 0x558c806883e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558c806057b0 0x558c8060ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff46f7fe700 0 client.15555 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.756+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x558c80600c80 0x558c80612f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4a0e57b80 1 --2- 172.21.15.77:0/874041239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558c80601640 0x558c806134a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/874041239 >> 172.21.15.77:0/874041239 conn(0x558c8052c3d0 msgr2=0x558c80608e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:28.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/874041239 shutdown_connections 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4a0e57b80 1 -- 172.21.15.77:0/874041239 wait complete. 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4a0e57b80 20 client.15555 trim_cache size 0 max 16384 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 33, 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 10, 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stdout: "5a8eccbd": { 2024-05-27T08:42:28.759 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 10, 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 130.0, 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:42:28.760 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:28.761 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:28.761 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:28.761 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:28.761 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:28.761 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 -- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff47c0b2aa0 msgr2=0x7ff47c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff47c0b2aa0 0x7ff47c0b4f60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff480009a10 tx=0x7ff480009190 comp rx=0 tx=0).stop 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 -- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4901073c0 msgr2=0x7ff490199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4901073c0 0x7ff490199360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b970 tx=0x7ff48800bc80 comp rx=0 tx=0).stop 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 -- 172.21.15.77:0/1599535410 shutdown_connections 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff47c0b2aa0 0x7ff47c0b4f60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4901073c0 0x7ff490199360 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff490102460 0x7ff4901988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 --2- 172.21.15.77:0/1599535410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff490102e20 0x7ff490198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 -- 172.21.15.77:0/1599535410 >> 172.21.15.77:0/1599535410 conn(0x7ff490079490 msgr2=0x7ff4901004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:28.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.757+0000 7ff4965c1700 1 -- 172.21.15.77:0/1599535410 shutdown_connections 2024-05-27T08:42:28.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:28.758+0000 7ff4965c1700 1 -- 172.21.15.77:0/1599535410 wait complete. 2024-05-27T08:42:29.528 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5a8eccbd', quiesce took 5.8 seconds. Will release after: 4.2 2024-05-27T08:42:29.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 3.33 MB/sec execute 23 sec latency 877.061 ms 2024-05-27T08:42:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:29 smithi077 ceph-mon[32007]: pgmap v129: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-05-27T08:42:30.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:29 smithi092 ceph-mon[36049]: pgmap v129: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-05-27T08:42:30.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:29 smithi179 ceph-mon[36481]: pgmap v129: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-05-27T08:42:30.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 3.19 MB/sec execute 24 sec latency 1877.140 ms 2024-05-27T08:42:31.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 3.06 MB/sec execute 25 sec latency 2877.219 ms 2024-05-27T08:42:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:31 smithi077 ceph-mon[32007]: pgmap v130: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-27T08:42:32.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:31 smithi092 ceph-mon[36049]: pgmap v130: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-27T08:42:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:31 smithi179 ceph-mon[36481]: pgmap v130: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-05-27T08:42:32.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 2.94 MB/sec execute 26 sec latency 3877.320 ms 2024-05-27T08:42:33.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 2.83 MB/sec execute 27 sec latency 4877.389 ms 2024-05-27T08:42:33.729 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:42:33.984 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:42:34.107 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:33 smithi077 ceph-mon[32007]: pgmap v131: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-27T08:42:34.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:33 smithi092 ceph-mon[36049]: pgmap v131: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-27T08:42:34.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:33 smithi179 ceph-mon[36481]: pgmap v131: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-05-27T08:42:34.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 2.73 MB/sec execute 28 sec latency 5877.462 ms 2024-05-27T08:42:35.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 2.64 MB/sec execute 29 sec latency 6877.824 ms 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.932+0000 7ff3b6a24700 1 -- 172.21.15.77:0/2420279353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 msgr2=0x7ff3b0110470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.932+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/2420279353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 0x7ff3b0110470 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff3a800b0d0 tx=0x7ff3a800b3e0 comp rx=0 tx=0).stop 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 -- 172.21.15.77:0/2420279353 shutdown_connections 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/2420279353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 0x7ff3b0110470 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/2420279353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b01037f0 0x7ff3b0103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/2420279353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b01041a0 0x7ff3b01081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 -- 172.21.15.77:0/2420279353 >> 172.21.15.77:0/2420279353 conn(0x7ff3b007a6a0 msgr2=0x7ff3b007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 -- 172.21.15.77:0/2420279353 shutdown_connections 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 -- 172.21.15.77:0/2420279353 wait complete. 2024-05-27T08:42:35.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 Processor -- start 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.933+0000 7ff3b6a24700 1 -- start start 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b6a24700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b01037f0 0x7ff3b019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b6a24700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b01041a0 0x7ff3b019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b6a24700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 0x7ff3b0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b6a24700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b01136c0 con 0x7ff3b01088f0 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b6a24700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0113540 con 0x7ff3b01037f0 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b6a24700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b0113840 con 0x7ff3b01041a0 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 0x7ff3b0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 0x7ff3b0199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53758/0 (socket says 172.21.15.77:53758) 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 -- 172.21.15.77:0/1724606984 learned_addr learned my addr 172.21.15.77:0/1724606984 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3affff700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b01037f0 0x7ff3b019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3af7fe700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b01041a0 0x7ff3b019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:35.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 -- 172.21.15.77:0/1724606984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b01041a0 msgr2=0x7ff3b019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b01041a0 0x7ff3b019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 -- 172.21.15.77:0/1724606984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b01037f0 msgr2=0x7ff3b019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b01037f0 0x7ff3b019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 -- 172.21.15.77:0/1724606984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b019a0e0 con 0x7ff3b01088f0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b4fc1700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 0x7ff3b0199980 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80095c0 tx=0x7ff3a8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3ad7fa700 1 -- 172.21.15.77:0/1724606984 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a801f070 con 0x7ff3b01088f0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b019a370 con 0x7ff3b01088f0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.934+0000 7ff3ad7fa700 1 -- 172.21.15.77:0/1724606984 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a8007e20 con 0x7ff3b01088f0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.935+0000 7ff3ad7fa700 1 -- 172.21.15.77:0/1724606984 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a801b4a0 con 0x7ff3b01088f0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.935+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b01a6690 con 0x7ff3b01088f0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.935+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b006a230 con 0x7ff3b01088f0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.937+0000 7ff3ad7fa700 1 -- 172.21.15.77:0/1724606984 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff3a8022070 con 0x7ff3b01088f0 2024-05-27T08:42:35.956 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.938+0000 7ff3ad7fa700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff3980b29f0 0x7ff3980b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.938+0000 7ff3affff700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff3980b29f0 0x7ff3980b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.938+0000 7ff3ad7fa700 1 -- 172.21.15.77:0/1724606984 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7ff3a80e3770 con 0x7ff3b01088f0 2024-05-27T08:42:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.938+0000 7ff3affff700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff3980b29f0 0x7ff3980b4eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff3b019aac0 tx=0x7ff3a000b410 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:35.957 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:35.942+0000 7ff3ad7fa700 1 -- 172.21.15.77:0/1724606984 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3a80ab1d0 con 0x7ff3b01088f0 2024-05-27T08:42:36.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:35 smithi077 ceph-mon[32007]: pgmap v132: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 799 KiB/s wr, 83 op/s 2024-05-27T08:42:36.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.197+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3b002e560 con 0x7ff3b01088f0 2024-05-27T08:42:36.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.198+0000 7ff3ad7fa700 1 -- 172.21.15.77:0/1724606984 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11934 (secure 0 0 0) 0x7ff3a80aaad0 con 0x7ff3b01088f0 2024-05-27T08:42:36.200 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:42:36.200 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:42:23.427101+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:42:36.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff3980b29f0 msgr2=0x7ff3980b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:36.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff3980b29f0 0x7ff3980b4eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff3b019aac0 tx=0x7ff3a000b410 comp rx=0 tx=0).stop 2024-05-27T08:42:36.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 msgr2=0x7ff3b0199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:36.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 0x7ff3b0199980 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80095c0 tx=0x7ff3a8007a20 comp rx=0 tx=0).stop 2024-05-27T08:42:36.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 shutdown_connections 2024-05-27T08:42:36.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7ff3980b29f0 0x7ff3980b4eb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:36.202 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3b01088f0 0x7ff3b0199980 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:36.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3b01037f0 0x7ff3b019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:36.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 --2- 172.21.15.77:0/1724606984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3b01041a0 0x7ff3b019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:36.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 >> 172.21.15.77:0/1724606984 conn(0x7ff3b007a6a0 msgr2=0x7ff3b01015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:36.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 shutdown_connections 2024-05-27T08:42:36.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:36.201+0000 7ff3b6a24700 1 -- 172.21.15.77:0/1724606984 wait complete. 2024-05-27T08:42:36.203 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 33 2024-05-27T08:42:36.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:35 smithi092 ceph-mon[36049]: pgmap v132: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 799 KiB/s wr, 83 op/s 2024-05-27T08:42:36.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:35 smithi179 ceph-mon[36481]: pgmap v132: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 799 KiB/s wr, 83 op/s 2024-05-27T08:42:36.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 2.55 MB/sec execute 30 sec latency 7877.862 ms 2024-05-27T08:42:36.952 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 5a8eccbd --release --await' 2024-05-27T08:42:36.952 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db --set-id 5a8eccbd --release --await 2024-05-27T08:42:37.193 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:42:37.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:36 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1724606984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:42:37.315 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:36 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1724606984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:42:37.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:36 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1724606984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:42:37.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 2.47 MB/sec execute 31 sec latency 8877.886 ms 2024-05-27T08:42:38.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:37 smithi092 ceph-mon[36049]: pgmap v133: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 799 KiB/s wr, 83 op/s 2024-05-27T08:42:38.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:37 smithi179 ceph-mon[36481]: pgmap v133: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 799 KiB/s wr, 83 op/s 2024-05-27T08:42:38.376 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:37 smithi077 ceph-mon[32007]: pgmap v133: 97 pgs: 97 active+clean; 71 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 799 KiB/s wr, 83 op/s 2024-05-27T08:42:38.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 103701 2.39 MB/sec execute 32 sec latency 9877.922 ms 2024-05-27T08:42:38.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/1054710090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d181028c0 msgr2=0x7f2d18110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/1054710090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d181028c0 0x7f2d18110330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2d1000b0a0 tx=0x7f2d1000b3b0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/1054710090 shutdown_connections 2024-05-27T08:42:38.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/1054710090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d181028c0 0x7f2d18110330 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/1054710090 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d18108620 0x7f2d18108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/1054710090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d18101f00 0x7f2d18102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/1054710090 >> 172.21.15.77:0/1054710090 conn(0x7f2d18077780 msgr2=0x7f2d18077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:38.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/1054710090 shutdown_connections 2024-05-27T08:42:38.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/1054710090 wait complete. 2024-05-27T08:42:38.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 Processor -- start 2024-05-27T08:42:38.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.914+0000 7f2d1f3ac700 1 -- start start 2024-05-27T08:42:38.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1f3ac700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d18101f00 0x7f2d1819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1f3ac700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d181028c0 0x7f2d1819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1f3ac700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d18108620 0x7f2d181998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1f3ac700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181137b0 con 0x7f2d18101f00 2024-05-27T08:42:38.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1f3ac700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18113630 con 0x7f2d181028c0 2024-05-27T08:42:38.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1f3ac700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18113930 con 0x7f2d18108620 2024-05-27T08:42:38.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d18101f00 0x7f2d1819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d18101f00 0x7f2d1819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53764/0 (socket says 172.21.15.77:53764) 2024-05-27T08:42:38.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 -- 172.21.15.77:0/3958178650 learned_addr learned my addr 172.21.15.77:0/3958178650 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:38.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1c947700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d181028c0 0x7f2d1819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d949700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d18108620 0x7f2d181998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 -- 172.21.15.77:0/3958178650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d18108620 msgr2=0x7f2d181998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d18108620 0x7f2d181998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 -- 172.21.15.77:0/3958178650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d181028c0 msgr2=0x7f2d1819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d181028c0 0x7f2d1819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 -- 172.21.15.77:0/3958178650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1819a050 con 0x7f2d18101f00 2024-05-27T08:42:38.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1c947700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d181028c0 0x7f2d1819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:42:38.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1d148700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d18101f00 0x7f2d1819f730 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d980 tx=0x7f2d1400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:38.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d0e7fc700 1 -- 172.21.15.77:0/3958178650 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14017030 con 0x7f2d18101f00 2024-05-27T08:42:38.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/3958178650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1819a340 con 0x7f2d18101f00 2024-05-27T08:42:38.922 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d0e7fc700 1 -- 172.21.15.77:0/3958178650 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d140044d0 con 0x7f2d18101f00 2024-05-27T08:42:38.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d0e7fc700 1 -- 172.21.15.77:0/3958178650 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14003aa0 con 0x7f2d18101f00 2024-05-27T08:42:38.923 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.915+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/3958178650 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d181a6600 con 0x7f2d18101f00 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.918+0000 7f2d0e7fc700 1 -- 172.21.15.77:0/3958178650 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2d1401a070 con 0x7f2d18101f00 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.918+0000 7f2d0e7fc700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f2d040b29f0 0x7f2d040b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.918+0000 7f2d1c947700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f2d040b29f0 0x7f2d040b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.918+0000 7f2d0e7fc700 1 -- 172.21.15.77:0/3958178650 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2d140e2030 con 0x7f2d18101f00 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d1c947700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f2d040b29f0 0x7f2d040b4eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005fd0 tx=0x7f2d08005e30 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d29c42b80 1 Processor -- start 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d29c42b80 1 -- start start 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c7673690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c766dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.931 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c767a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c6c75ccd90 con 0x55c6c766d8e0 2024-05-27T08:42:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c6c75bc930 con 0x55c6c766e2a0 2024-05-27T08:42:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c6c75c0970 con 0x55c6c76732b0 2024-05-27T08:42:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d1c947700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c766dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d1c947700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c766dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53774/0 (socket says 172.21.15.77:53774) 2024-05-27T08:42:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.919+0000 7f2d1c947700 1 -- 172.21.15.77:0/4156516493 learned_addr learned my addr 172.21.15.77:0/4156516493 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:38.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1d949700 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c767a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1c947700 1 -- 172.21.15.77:0/4156516493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 msgr2=0x55c6c7673690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1d148700 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c7673690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1c947700 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c7673690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1c947700 1 -- 172.21.15.77:0/4156516493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 msgr2=0x55c6c767a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:38.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1c947700 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c767a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1c947700 1 -- 172.21.15.77:0/4156516493 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c750c9d0 con 0x55c6c766d8e0 2024-05-27T08:42:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1d949700 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c767a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:42:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1d148700 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c7673690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:42:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d1c947700 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c766dd60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800eef0 tx=0x7f2d08011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98edbaa05b3a7c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d037fe700 1 -- 172.21.15.77:0/4156516493 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08021070 con 0x55c6c766d8e0 2024-05-27T08:42:38.934 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d037fe700 1 -- 172.21.15.77:0/4156516493 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d08010e50 con 0x55c6c766d8e0 2024-05-27T08:42:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.920+0000 7f2d037fe700 1 -- 172.21.15.77:0/4156516493 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800f7c0 con 0x55c6c766d8e0 2024-05-27T08:42:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 -- 172.21.15.77:0/4156516493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 msgr2=0x55c6c766dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c766dd60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800eef0 tx=0x7f2d08011000 comp rx=0 tx=0).stop 2024-05-27T08:42:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 -- 172.21.15.77:0/4156516493 shutdown_connections 2024-05-27T08:42:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c767a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.935 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c7673690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/4156516493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c766dd60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 -- 172.21.15.77:0/4156516493 >> 172.21.15.77:0/4156516493 conn(0x55c6c759b180 msgr2=0x55c6c76681f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 -- 172.21.15.77:0/4156516493 shutdown_connections 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 -- 172.21.15.77:0/4156516493 wait complete. 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 -- start start 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.921+0000 7f2d29c42b80 1 Processor -- start 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766d8e0 0x55c6c7679de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 0x55c6c76770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.936 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c76732b0 0x55c6c7677620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c6c74c30a0 con 0x55c6c76732b0 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c6c755e8c0 con 0x55c6c766d8e0 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c6c767a6e0 con 0x55c6c766e2a0 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d29c42b80 10 client.0 _ll_register_callbacks cb 0x55c6c7557a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c76732b0 0x55c6c7677620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c76732b0 0x55c6c7677620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53788/0 (socket says 172.21.15.77:53788) 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 -- 172.21.15.77:0/1316401726 learned_addr learned my addr 172.21.15.77:0/1316401726 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766d8e0 0x55c6c7679de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 -- 172.21.15.77:0/1316401726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 msgr2=0x55c6c76770e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 0x55c6c76770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 -- 172.21.15.77:0/1316401726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766d8e0 msgr2=0x55c6c7679de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766d8e0 0x55c6c7679de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 -- 172.21.15.77:0/1316401726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c755b790 con 0x55c6c76732b0 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d1d949700 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c76732b0 0x55c6c7677620 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2d10004d00 tx=0x7f2d10004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1462a809e0820774 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.922+0000 7f2d00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1316401726 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d100076e0 con 0x55c6c76732b0 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1316401726 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2d10007880 con 0x55c6c76732b0 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1316401726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d08010e50 con 0x55c6c76732b0 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1316401726 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2d10007a80 con 0x55c6c76732b0 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 10 client.15573 fetch_fsmap learned FSMap version 33 2024-05-27T08:42:38.938 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1316401726 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c6c7684dd0 con 0x55c6c76732b0 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1316401726 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d08010e50 con 0x55c6c76732b0 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1316401726 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f2d10011070 con 0x55c6c76732b0 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 10 client.15573 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 20 client.15573 populate_metadata read hostname 'smithi077' 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c6c7687480 0x55c6c76898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 4 client.15573 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.923+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1316401726 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6c7639ae0 con 0x55c6c7687480 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.924+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c6c7687480 0x55c6c76898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.924+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c6c7687480 0x55c6c76898a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7633f107bf5c85ce server_cookie=ce0b57725d0a4270 in_seq=0 out_seq=0 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.924+0000 7f2d00ff9700 10 client.15573 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:42:38.939 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1316401726 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c6c7639ae0 con 0x55c6c7687480 2024-05-27T08:42:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d00ff9700 10 client.15573 handle_command_reply: tid=0 2024-05-27T08:42:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 client.15573 shutdown 2024-05-27T08:42:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1316401726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c76732b0 msgr2=0x55c6c7677620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c76732b0 0x55c6c7677620 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2d10004d00 tx=0x7f2d10004d30 comp rx=0 tx=0).stop 2024-05-27T08:42:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1316401726 shutdown_connections 2024-05-27T08:42:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c6c7687480 0x55c6c76898a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.940 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c76732b0 0x55c6c7677620 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766d8e0 0x55c6c7679de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1316401726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 0x55c6c76770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1316401726 >> 172.21.15.77:0/1316401726 conn(0x55c6c759b180 msgr2=0x55c6c7662ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d00ff9700 0 client.15573 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:42:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1316401726 shutdown_connections 2024-05-27T08:42:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1316401726 wait complete. 2024-05-27T08:42:38.941 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.925+0000 7f2d29c42b80 20 client.15573 trim_cache size 0 max 16384 2024-05-27T08:42:38.945 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d29c42b80 1 Processor -- start 2024-05-27T08:42:38.946 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d29c42b80 1 -- start start 2024-05-27T08:42:38.946 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:38 smithi077 ceph-mon[32007]: pgmap v134: 97 pgs: 97 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 821 KiB/s wr, 94 op/s 2024-05-27T08:42:38.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c7672900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c7679920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.947 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c766fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181add10 con 0x55c6c766d8e0 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181af910 con 0x55c6c766e2a0 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181ae3e0 con 0x55c6c76732b0 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d1d148700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c7672900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d1d148700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c7672900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53798/0 (socket says 172.21.15.77:53798) 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d1d148700 1 -- 172.21.15.77:0/1575938053 learned_addr learned my addr 172.21.15.77:0/1575938053 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.945+0000 7f2d1d949700 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c766fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d1c947700 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c7679920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d1d148700 1 -- 172.21.15.77:0/1575938053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 msgr2=0x55c6c766fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c766fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d1d148700 1 -- 172.21.15.77:0/1575938053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 msgr2=0x55c6c7679920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c7679920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.948 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d1d148700 1 -- 172.21.15.77:0/1575938053 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6c7672e40 con 0x55c6c766d8e0 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d1d949700 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c766fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c7672900 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2d140a9020 tx=0x7f2d14009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f7f26322b6c09e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d02ffd700 1 -- 172.21.15.77:0/1575938053 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140ad6d0 con 0x55c6c766d8e0 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d02ffd700 1 -- 172.21.15.77:0/1575938053 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14014070 con 0x55c6c766d8e0 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.946+0000 7f2d02ffd700 1 -- 172.21.15.77:0/1575938053 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140b1f50 con 0x55c6c766d8e0 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1575938053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 msgr2=0x55c6c7672900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c7672900 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2d140a9020 tx=0x7f2d14009ec0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1575938053 shutdown_connections 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c76732b0 0x55c6c766fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c7672900 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1575938053 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c766e2a0 0x55c6c7679920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1575938053 >> 172.21.15.77:0/1575938053 conn(0x55c6c759b180 msgr2=0x55c6c766d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:38.949 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1575938053 shutdown_connections 2024-05-27T08:42:38.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1575938053 wait complete. 2024-05-27T08:42:38.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- start start 2024-05-27T08:42:38.950 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 Processor -- start 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c767ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 0x55c6c767a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c76732b0 0x55c6c767ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181af180 con 0x55c6c766d8e0 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181aeab0 con 0x55c6c76732b0 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.947+0000 7f2d29c42b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d181abcb0 con 0x55c6c766e2a0 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d29c42b80 10 client.0 _ll_register_callbacks cb 0x55c6c7557a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c767ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.952 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c767ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:53812/0 (socket says 172.21.15.77:53812) 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 -- 172.21.15.77:0/1590050862 learned_addr learned my addr 172.21.15.77:0/1590050862 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1c947700 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 0x55c6c767a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d949700 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c76732b0 0x55c6c767ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 -- 172.21.15.77:0/1590050862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 msgr2=0x55c6c767a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 0x55c6c767a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 -- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c76732b0 msgr2=0x55c6c767ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c76732b0 0x55c6c767ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 -- 172.21.15.77:0/1590050862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6c767b5b0 con 0x55c6c766d8e0 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d949700 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c76732b0 0x55c6c767ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d1d148700 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c767ffc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2d140df840 tx=0x7f2d140a93e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3900c7b1e40746 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1590050862 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140a9790 con 0x55c6c766d8e0 2024-05-27T08:42:38.953 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1590050862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14014070 con 0x55c6c766d8e0 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1590050862 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f2d140b1cf0 con 0x55c6c766d8e0 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1590050862 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2d140b1ef0 con 0x55c6c766d8e0 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d29c42b80 10 client.15579 fetch_fsmap learned FSMap version 33 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.948+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1590050862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c6c76f3110 con 0x55c6c766d8e0 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1590050862 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14014070 con 0x55c6c766d8e0 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1590050862 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f2d14024060 con 0x55c6c766d8e0 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d29c42b80 10 client.15579 fetch_fsmap finished waiting for FSMap version 33 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d29c42b80 10 client.15579 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d29c42b80 20 client.15579 populate_metadata read hostname 'smithi077' 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c6c76f3860 0x55c6c76f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d29c42b80 4 client.15579 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5a8eccbd", "release": true, "await": true}] 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1590050862 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5a8eccbd", "release": true, "await": true}) -- 0x55c6c75cece0 con 0x55c6c76f3860 2024-05-27T08:42:38.954 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.949+0000 7f2d1c947700 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c6c76f3860 0x55c6c76f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:42:38.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.950+0000 7f2d00ff9700 10 client.15579 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:42:38.955 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.950+0000 7f2d1c947700 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c6c76f3860 0x55c6c76f5c80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b3235bd8e41edc server_cookie=a6ee655f3ecba2e1 in_seq=0 out_seq=0 2024-05-27T08:42:38.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d00ff9700 1 -- 172.21.15.77:0/1590050862 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c6c75cece0 con 0x55c6c76f3860 2024-05-27T08:42:38.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d00ff9700 10 client.15579 handle_command_reply: tid=0 2024-05-27T08:42:38.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 client.15579 shutdown 2024-05-27T08:42:38.959 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1590050862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 msgr2=0x55c6c767ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c767ffc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2d140df840 tx=0x7f2d140a93e0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1590050862 shutdown_connections 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c6c76f3860 0x55c6c76f5c80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c6c76732b0 0x55c6c767ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c6c766d8e0 0x55c6c767ffc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 --2- 172.21.15.77:0/1590050862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6c766e2a0 0x55c6c767a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1590050862 >> 172.21.15.77:0/1590050862 conn(0x55c6c759b180 msgr2=0x55c6c7662410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.958+0000 7f2d00ff9700 0 client.15579 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1590050862 shutdown_connections 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d29c42b80 1 -- 172.21.15.77:0/1590050862 wait complete. 2024-05-27T08:42:38.960 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d29c42b80 20 client.15579 trim_cache size 0 max 16384 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 33, 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 12, 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "5a8eccbd": { 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 12, 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:42:38.961 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 130.0, 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:38.962 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:38.963 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:42:38.963 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:42:38.963 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f2d040b29f0 msgr2=0x7f2d040b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f2d040b29f0 0x7f2d040b4eb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2d08005fd0 tx=0x7f2d08005e30 comp rx=0 tx=0).stop 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d18101f00 msgr2=0x7f2d1819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d18101f00 0x7f2d1819f730 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400d980 tx=0x7f2d1400dd40 comp rx=0 tx=0).stop 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/3958178650 shutdown_connections 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f2d040b29f0 0x7f2d040b4eb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d18108620 0x7f2d181998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d18101f00 0x7f2d1819f730 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 --2- 172.21.15.77:0/3958178650 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2d181028c0 0x7f2d1819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.959+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/3958178650 >> 172.21.15.77:0/3958178650 conn(0x7f2d18077780 msgr2=0x7f2d18100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.960+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/3958178650 shutdown_connections 2024-05-27T08:42:38.964 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:42:38.960+0000 7f2d1f3ac700 1 -- 172.21.15.77:0/3958178650 wait complete. 2024-05-27T08:42:39.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:38 smithi092 ceph-mon[36049]: pgmap v134: 97 pgs: 97 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 821 KiB/s wr, 94 op/s 2024-05-27T08:42:39.256 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:38 smithi179 ceph-mon[36481]: pgmap v134: 97 pgs: 97 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 821 KiB/s wr, 94 op/s 2024-05-27T08:42:39.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 105365 2.52 MB/sec execute 33 sec latency 10241.995 ms 2024-05-27T08:42:39.797 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5a8eccbd', total seconds elapsed: 16.1 2024-05-27T08:42:40.596 INFO:tasks.workunit.client.0.smithi077.stdout: 1 106930 2.80 MB/sec execute 34 sec latency 20.814 ms 2024-05-27T08:42:41.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 107173 2.73 MB/sec execute 35 sec latency 784.783 ms 2024-05-27T08:42:41.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:41 smithi179 ceph-mon[36481]: pgmap v135: 97 pgs: 97 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 304 KiB/s wr, 32 op/s 2024-05-27T08:42:41.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:41 smithi077 ceph-mon[32007]: pgmap v135: 97 pgs: 97 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 304 KiB/s wr, 32 op/s 2024-05-27T08:42:41.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:41 smithi092 ceph-mon[36049]: pgmap v135: 97 pgs: 97 active+clean; 68 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 304 KiB/s wr, 32 op/s 2024-05-27T08:42:42.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 107173 2.65 MB/sec execute 36 sec latency 1784.883 ms 2024-05-27T08:42:43.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 108311 2.72 MB/sec execute 37 sec latency 2279.074 ms 2024-05-27T08:42:43.606 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:43 smithi179 ceph-mon[36481]: pgmap v136: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 785 KiB/s wr, 90 op/s 2024-05-27T08:42:43.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:43 smithi077 ceph-mon[32007]: pgmap v136: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 785 KiB/s wr, 90 op/s 2024-05-27T08:42:43.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:43 smithi092 ceph-mon[36049]: pgmap v136: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 785 KiB/s wr, 90 op/s 2024-05-27T08:42:44.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 110385 3.01 MB/sec execute 38 sec latency 10.461 ms 2024-05-27T08:42:45.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 110791 2.94 MB/sec execute 39 sec latency 616.108 ms 2024-05-27T08:42:45.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:45 smithi179 ceph-mon[36481]: pgmap v137: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 502 KiB/s wr, 68 op/s 2024-05-27T08:42:45.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:45 smithi077 ceph-mon[32007]: pgmap v137: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 502 KiB/s wr, 68 op/s 2024-05-27T08:42:45.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:45 smithi092 ceph-mon[36049]: pgmap v137: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 502 KiB/s wr, 68 op/s 2024-05-27T08:42:46.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 110791 2.87 MB/sec execute 40 sec latency 1616.160 ms 2024-05-27T08:42:47.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 110791 2.80 MB/sec execute 41 sec latency 2616.261 ms 2024-05-27T08:42:47.605 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:47 smithi179 ceph-mon[36481]: pgmap v138: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 502 KiB/s wr, 68 op/s 2024-05-27T08:42:47.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:47 smithi077 ceph-mon[32007]: pgmap v138: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 502 KiB/s wr, 68 op/s 2024-05-27T08:42:47.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:47 smithi092 ceph-mon[36049]: pgmap v138: 97 pgs: 97 active+clean; 70 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 502 KiB/s wr, 68 op/s 2024-05-27T08:42:48.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 112335 2.89 MB/sec execute 42 sec latency 3113.251 ms 2024-05-27T08:42:49.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 113976 3.10 MB/sec execute 43 sec latency 12.351 ms 2024-05-27T08:42:49.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:49 smithi077 ceph-mon[32007]: pgmap v139: 97 pgs: 97 active+clean; 75 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 996 KiB/s wr, 124 op/s 2024-05-27T08:42:49.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:49 smithi092 ceph-mon[36049]: pgmap v139: 97 pgs: 97 active+clean; 75 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 996 KiB/s wr, 124 op/s 2024-05-27T08:42:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:49 smithi179 ceph-mon[36481]: pgmap v139: 97 pgs: 97 active+clean; 75 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 996 KiB/s wr, 124 op/s 2024-05-27T08:42:50.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 114266 3.04 MB/sec execute 44 sec latency 757.542 ms 2024-05-27T08:42:51.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 114266 2.97 MB/sec execute 45 sec latency 1757.625 ms 2024-05-27T08:42:51.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:51 smithi077 ceph-mon[32007]: pgmap v140: 97 pgs: 97 active+clean; 75 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 974 KiB/s wr, 113 op/s 2024-05-27T08:42:51.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:51 smithi092 ceph-mon[36049]: pgmap v140: 97 pgs: 97 active+clean; 75 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 974 KiB/s wr, 113 op/s 2024-05-27T08:42:51.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:51 smithi179 ceph-mon[36481]: pgmap v140: 97 pgs: 97 active+clean; 75 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 974 KiB/s wr, 113 op/s 2024-05-27T08:42:52.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 114266 2.91 MB/sec execute 46 sec latency 2757.752 ms 2024-05-27T08:42:53.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 115442 2.95 MB/sec execute 47 sec latency 3256.386 ms 2024-05-27T08:42:53.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:53 smithi077 ceph-mon[32007]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-27T08:42:53.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:53 smithi092 ceph-mon[36049]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-27T08:42:53.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:53 smithi179 ceph-mon[36481]: pgmap v141: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-05-27T08:42:54.597 INFO:tasks.workunit.client.0.smithi077.stdout: 1 117486 3.18 MB/sec execute 48 sec latency 12.254 ms 2024-05-27T08:42:55.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 119545 3.26 MB/sec execute 49 sec latency 36.678 ms 2024-05-27T08:42:55.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:55 smithi077 ceph-mon[32007]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 962 KiB/s wr, 108 op/s 2024-05-27T08:42:55.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:55 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:42:55.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:55 smithi092 ceph-mon[36049]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 962 KiB/s wr, 108 op/s 2024-05-27T08:42:55.735 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:55 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:42:55.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:55 smithi179 ceph-mon[36481]: pgmap v142: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 962 KiB/s wr, 108 op/s 2024-05-27T08:42:55.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:55 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:42:56.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 121044 3.43 MB/sec execute 50 sec latency 95.182 ms 2024-05-27T08:42:57.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 121356 3.37 MB/sec execute 51 sec latency 709.813 ms 2024-05-27T08:42:57.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:57 smithi179 ceph-mon[36481]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 962 KiB/s wr, 108 op/s 2024-05-27T08:42:57.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:57 smithi077 ceph-mon[32007]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 962 KiB/s wr, 108 op/s 2024-05-27T08:42:57.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:57 smithi092 ceph-mon[36049]: pgmap v143: 97 pgs: 97 active+clean; 77 MiB data, 779 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 962 KiB/s wr, 108 op/s 2024-05-27T08:42:58.369 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-27T08:42:58.370 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-05-27T08:42:58.370 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:42:58.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 121424 3.31 MB/sec execute 52 sec latency 1206.640 ms 2024-05-27T08:42:58.628 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:42:59.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 121424 3.25 MB/sec execute 53 sec latency 1453.773 ms 2024-05-27T08:42:59.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:42:59 smithi077 ceph-mon[32007]: pgmap v144: 97 pgs: 97 active+clean; 78 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-05-27T08:42:59.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:42:59 smithi179 ceph-mon[36481]: pgmap v144: 97 pgs: 97 active+clean; 78 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-05-27T08:42:59.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:42:59 smithi092 ceph-mon[36049]: pgmap v144: 97 pgs: 97 active+clean; 78 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 204 op/s 2024-05-27T08:43:00.273 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/3646694088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4108920 msgr2=0x7f0df41104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/3646694088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4108920 0x7f0df41104a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00b0d0 tx=0x7f0dec00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/3646694088 shutdown_connections 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/3646694088 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4108920 0x7f0df41104a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/3646694088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df4103850 0x7f0df4103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/3646694088 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0df4104200 0x7f0df41081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/3646694088 >> 172.21.15.77:0/3646694088 conn(0x7f0df407a620 msgr2=0x7f0df407aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/3646694088 shutdown_connections 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.271+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/3646694088 wait complete. 2024-05-27T08:43:00.274 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0dfa5d6700 1 Processor -- start 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0dfa5d6700 1 -- start start 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0dfa5d6700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4103850 0x7f0df419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0dfa5d6700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0df4104200 0x7f0df419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0dfa5d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df4108920 0x7f0df4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0dfa5d6700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4113710 con 0x7f0df4103850 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0dfa5d6700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4113590 con 0x7f0df4104200 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0dfa5d6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0df4113890 con 0x7f0df4108920 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0df3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4103850 0x7f0df419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0debfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0df4104200 0x7f0df419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0df3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4103850 0x7f0df419f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57338/0 (socket says 172.21.15.77:57338) 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.272+0000 7f0df3fff700 1 -- 172.21.15.77:0/2095692833 learned_addr learned my addr 172.21.15.77:0/2095692833 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0df8b73700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df4108920 0x7f0df4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:00.275 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0debfff700 1 -- 172.21.15.77:0/2095692833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df4108920 msgr2=0x7f0df4199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:00.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0debfff700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df4108920 0x7f0df4199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0debfff700 1 -- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4103850 msgr2=0x7f0df419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:00.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0debfff700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4103850 0x7f0df419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0debfff700 1 -- 172.21.15.77:0/2095692833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df419a0d0 con 0x7f0df4104200 2024-05-27T08:43:00.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0df3fff700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4103850 0x7f0df419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:43:00.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0debfff700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0df4104200 0x7f0df419fcf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e9f0 tx=0x7f0ddc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0df1ffb700 1 -- 172.21.15.77:0/2095692833 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00c9c0 con 0x7f0df4104200 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df419a3c0 con 0x7f0df4104200 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df4075770 con 0x7f0df4104200 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0df1ffb700 1 -- 172.21.15.77:0/2095692833 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ddc00cb60 con 0x7f0df4104200 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.273+0000 7f0df1ffb700 1 -- 172.21.15.77:0/2095692833 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc018680 con 0x7f0df4104200 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.274+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd4005330 con 0x7f0df4104200 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.276+0000 7f0df1ffb700 1 -- 172.21.15.77:0/2095692833 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0ddc018820 con 0x7f0df4104200 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.276+0000 7f0df1ffb700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f0de00b2a70 0x7f0de00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:00.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.277+0000 7f0df3fff700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f0de00b2a70 0x7f0de00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:00.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.277+0000 7f0df1ffb700 1 -- 172.21.15.77:0/2095692833 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f0ddc0e2940 con 0x7f0df4104200 2024-05-27T08:43:00.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.277+0000 7f0df3fff700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f0de00b2a70 0x7f0de00b4f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0df419aab0 tx=0x7f0de400b480 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:00.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.281+0000 7f0df1ffb700 1 -- 172.21.15.77:0/2095692833 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ddc0aa330 con 0x7f0df4104200 2024-05-27T08:43:00.565 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.565+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dd4005620 con 0x7f0df4104200 2024-05-27T08:43:00.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.566+0000 7f0df1ffb700 1 -- 172.21.15.77:0/2095692833 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11927 (secure 0 0 0) 0x7f0ddc0b23b0 con 0x7f0df4104200 2024-05-27T08:43:00.566 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:43:00.566 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:42:58.669173+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24418,24434,24410,24412,24433]},"id":1}]} 2024-05-27T08:43:00.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f0de00b2a70 msgr2=0x7f0de00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:00.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f0de00b2a70 0x7f0de00b4f30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0df419aab0 tx=0x7f0de400b480 comp rx=0 tx=0).stop 2024-05-27T08:43:00.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0df4104200 msgr2=0x7f0df419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:00.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0df4104200 0x7f0df419fcf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e9f0 tx=0x7f0ddc00ed00 comp rx=0 tx=0).stop 2024-05-27T08:43:00.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 shutdown_connections 2024-05-27T08:43:00.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f0de00b2a70 0x7f0de00b4f30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0df4108920 0x7f0df4199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0df4103850 0x7f0df419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 --2- 172.21.15.77:0/2095692833 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0df4104200 0x7f0df419fcf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:00.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 >> 172.21.15.77:0/2095692833 conn(0x7f0df407a620 msgr2=0x7f0df4109a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:00.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 shutdown_connections 2024-05-27T08:43:00.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:00.568+0000 7f0dfa5d6700 1 -- 172.21.15.77:0/2095692833 wait complete. 2024-05-27T08:43:00.570 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-05-27T08:43:00.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 121821 3.22 MB/sec execute 54 sec latency 1922.805 ms 2024-05-27T08:43:00.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:00 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:43:00.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:00 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2095692833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:00.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:00 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:43:00.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:00 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2095692833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:00.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:00 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:43:00.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:00 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2095692833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:01.375 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-05-27T08:43:01.375 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-05-27T08:43:01.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 124406 3.46 MB/sec execute 55 sec latency 10.705 ms 2024-05-27T08:43:01.629 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:01 smithi077 ceph-mon[32007]: pgmap v145: 97 pgs: 97 active+clean; 78 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-27T08:43:01.629 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:43:01.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:01 smithi179 ceph-mon[36481]: pgmap v145: 97 pgs: 97 active+clean; 78 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-27T08:43:01.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:01 smithi092 ceph-mon[36049]: pgmap v145: 97 pgs: 97 active+clean; 78 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-05-27T08:43:02.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 124973 3.41 MB/sec execute 56 sec latency 419.366 ms 2024-05-27T08:43:03.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 -- 172.21.15.77:0/2506262019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 msgr2=0x7f3b40108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 --2- 172.21.15.77:0/2506262019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 0x7f3b40108180 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b340099c0 tx=0x7f3b34009cd0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 -- 172.21.15.77:0/2506262019 shutdown_connections 2024-05-27T08:43:03.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 --2- 172.21.15.77:0/2506262019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b40108860 0x7f3b40110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 --2- 172.21.15.77:0/2506262019 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b4006beb0 0x7f3b4006c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 --2- 172.21.15.77:0/2506262019 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 0x7f3b40108180 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.498 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 -- 172.21.15.77:0/2506262019 >> 172.21.15.77:0/2506262019 conn(0x7f3b40078490 msgr2=0x7f3b400788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:03.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 -- 172.21.15.77:0/2506262019 shutdown_connections 2024-05-27T08:43:03.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.497+0000 7f3b45af9700 1 -- 172.21.15.77:0/2506262019 wait complete. 2024-05-27T08:43:03.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b45af9700 1 Processor -- start 2024-05-27T08:43:03.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b45af9700 1 -- start start 2024-05-27T08:43:03.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b45af9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4006beb0 0x7f3b4019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b45af9700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 0x7f3b4019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b45af9700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b40108860 0x7f3b40199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b45af9700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40113790 con 0x7f3b4006c860 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b45af9700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40113610 con 0x7f3b40108860 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b45af9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40113910 con 0x7f3b4006beb0 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4006beb0 0x7f3b4019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 0x7f3b4019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4006beb0 0x7f3b4019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.77:50874/0 (socket says 172.21.15.77:50874) 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b3ffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b40108860 0x7f3b40199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/3510994555 learned_addr learned my addr 172.21.15.77:0/3510994555 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.498+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 0x7f3b4019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57354/0 (socket says 172.21.15.77:57354) 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/3510994555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b40108860 msgr2=0x7f3b40199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b40108860 0x7f3b40199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 msgr2=0x7f3b4019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 0x7f3b4019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/3510994555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4019a0b0 con 0x7f3b4006beb0 2024-05-27T08:43:03.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3effd700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 0x7f3b4019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:43:03.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4006beb0 0x7f3b4019f790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000e9a0 tx=0x7f3b3000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:03.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3cff9700 1 -- 172.21.15.77:0/3510994555 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000ca10 con 0x7f3b4006beb0 2024-05-27T08:43:03.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b45af9700 1 -- 172.21.15.77:0/3510994555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4019a3a0 con 0x7f3b4006beb0 2024-05-27T08:43:03.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3cff9700 1 -- 172.21.15.77:0/3510994555 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b3000cbb0 con 0x7f3b4006beb0 2024-05-27T08:43:03.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b3cff9700 1 -- 172.21.15.77:0/3510994555 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30018680 con 0x7f3b4006beb0 2024-05-27T08:43:03.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.499+0000 7f3b45af9700 1 -- 172.21.15.77:0/3510994555 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b401a6660 con 0x7f3b4006beb0 2024-05-27T08:43:03.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.501+0000 7f3b3cff9700 1 -- 172.21.15.77:0/3510994555 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3b30010430 con 0x7f3b4006beb0 2024-05-27T08:43:03.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.502+0000 7f3b3cff9700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3b280b2aa0 0x7f3b280b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.502+0000 7f3b3effd700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3b280b2aa0 0x7f3b280b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.502+0000 7f3b3cff9700 1 -- 172.21.15.77:0/3510994555 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b300e27c0 con 0x7f3b4006beb0 2024-05-27T08:43:03.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b3effd700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3b280b2aa0 0x7f3b280b4f60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b34000c00 tx=0x7f3b34005fb0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b5038fb80 1 Processor -- start 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b5038fb80 1 -- start start 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac983720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac97ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac98a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5629ac8dce20 con 0x5629ac97d970 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5629ac8cc9c0 con 0x5629ac983340 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.503+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5629ac8d0a00 con 0x5629ac97e330 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac983720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac97ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.505 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac983720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:44086/0 (socket says 172.21.15.77:44086) 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/2596078770 learned_addr learned my addr 172.21.15.77:0/2596078770 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3ffff700 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac98a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3effd700 1 -- 172.21.15.77:0/2596078770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 msgr2=0x5629ac98a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3effd700 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac98a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3effd700 1 -- 172.21.15.77:0/2596078770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 msgr2=0x5629ac983720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3effd700 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac983720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3effd700 1 -- 172.21.15.77:0/2596078770 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ac81ca60 con 0x5629ac97d970 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac983720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b3effd700 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac97ddf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b580 tx=0x7f3b34020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241340ac47a95e05 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b25ffb700 1 -- 172.21.15.77:0/2596078770 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34005690 con 0x5629ac97d970 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b25ffb700 1 -- 172.21.15.77:0/2596078770 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b3400bdf0 con 0x5629ac97d970 2024-05-27T08:43:03.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.504+0000 7f3b25ffb700 1 -- 172.21.15.77:0/2596078770 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34024ac0 con 0x5629ac97d970 2024-05-27T08:43:03.511 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2596078770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 msgr2=0x5629ac97ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac97ddf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b580 tx=0x7f3b34020000 comp rx=0 tx=0).stop 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2596078770 shutdown_connections 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac98a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac983720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2596078770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac97ddf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2596078770 >> 172.21.15.77:0/2596078770 conn(0x5629ac8ab210 msgr2=0x5629ac978280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2596078770 shutdown_connections 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.505+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2596078770 wait complete. 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.506+0000 7f3b5038fb80 1 -- start start 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.506+0000 7f3b5038fb80 1 Processor -- start 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac989e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.512 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac987110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac987650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5629ac7d3130 con 0x5629ac97d970 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x5629ac86e950 con 0x5629ac983340 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5629ac98a6b0 con 0x5629ac97e330 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b5038fb80 10 client.0 _ll_register_callbacks cb 0x5629ac867ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac989e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac989e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57360/0 (socket says 172.21.15.77:57360) 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/2443572299 learned_addr learned my addr 172.21.15.77:0/2443572299 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3effd700 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac987110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3ffff700 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac987650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/2443572299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 msgr2=0x5629ac987110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac987110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/2443572299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 msgr2=0x5629ac987650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac987650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/2443572299 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ac86b820 con 0x5629ac97d970 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac989e10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009230 tx=0x7f3b300a9f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db10c01ee52ad728 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b1effd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b1effd700 1 -- 172.21.15.77:0/2443572299 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300a5040 con 0x5629ac97d970 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b1effd700 1 -- 172.21.15.77:0/2443572299 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b300a8ec0 con 0x5629ac97d970 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.507+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2443572299 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b3400bdf0 con 0x5629ac97d970 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b1effd700 1 -- 172.21.15.77:0/2443572299 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3b300a9560 con 0x5629ac97d970 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b5038fb80 10 client.15597 fetch_fsmap learned FSMap version 35 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2443572299 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5629ac994e70 con 0x5629ac97d970 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b1effd700 1 -- 172.21.15.77:0/2443572299 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b3400bdf0 con 0x5629ac97d970 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b1effd700 1 -- 172.21.15.77:0/2443572299 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f3b300cc070 con 0x5629ac97d970 2024-05-27T08:43:03.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b5038fb80 10 client.15597 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b5038fb80 10 client.15597 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b5038fb80 20 client.15597 populate_metadata read hostname 'smithi077' 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5629ac9974b0 0x5629ac9998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b5038fb80 4 client.15597 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2443572299 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629ac949b70 con 0x5629ac9974b0 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b3effd700 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5629ac9974b0 0x5629ac9998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.508+0000 7f3b1effd700 10 client.15597 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.509+0000 7f3b3effd700 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5629ac9974b0 0x5629ac9998a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99ebd5682c95dce server_cookie=c419cb306271a583 in_seq=0 out_seq=0 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b1effd700 1 -- 172.21.15.77:0/2443572299 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5629ac949b70 con 0x5629ac9974b0 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b1effd700 10 client.15597 handle_command_reply: tid=0 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 client.15597 shutdown 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2443572299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 msgr2=0x5629ac989e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac989e10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009230 tx=0x7f3b300a9f90 comp rx=0 tx=0).stop 2024-05-27T08:43:03.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2443572299 shutdown_connections 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5629ac9974b0 0x5629ac9998a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac987650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac989e10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b1effd700 0 client.15597 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2443572299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac987110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2443572299 >> 172.21.15.77:0/2443572299 conn(0x5629ac8ab210 msgr2=0x5629ac972d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2443572299 shutdown_connections 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2443572299 wait complete. 2024-05-27T08:43:03.516 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.510+0000 7f3b5038fb80 20 client.15597 trim_cache size 0 max 16384 2024-05-27T08:43:03.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b5038fb80 1 Processor -- start 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b5038fb80 1 -- start start 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac972bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac989950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac97fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401adc60 con 0x5629ac97d970 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401af7a0 con 0x5629ac983340 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401ae330 con 0x5629ac97e330 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac972bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac972bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57372/0 (socket says 172.21.15.77:57372) 2024-05-27T08:43:03.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/3681385175 learned_addr learned my addr 172.21.15.77:0/3681385175 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3ffff700 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac97fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3effd700 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac989950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/3681385175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 msgr2=0x5629ac989950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac989950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/3681385175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 msgr2=0x5629ac97fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac97fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.534+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/3681385175 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629ac985680 con 0x5629ac97d970 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.535+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac972bb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000e970 tx=0x7f3b300b2700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984295f61601f85 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.535+0000 7f3b257fa700 1 -- 172.21.15.77:0/3681385175 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300a93a0 con 0x5629ac97d970 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.535+0000 7f3b257fa700 1 -- 172.21.15.77:0/3681385175 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b300b2040 con 0x5629ac97d970 2024-05-27T08:43:03.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.535+0000 7f3b257fa700 1 -- 172.21.15.77:0/3681385175 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3006f2b0 con 0x5629ac97d970 2024-05-27T08:43:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 -- 172.21.15.77:0/3681385175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 msgr2=0x5629ac972bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac972bb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000e970 tx=0x7f3b300b2700 comp rx=0 tx=0).stop 2024-05-27T08:43:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 -- 172.21.15.77:0/3681385175 shutdown_connections 2024-05-27T08:43:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac983340 0x5629ac97fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.540 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97d970 0x5629ac972bb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/3681385175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac97e330 0x5629ac989950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 -- 172.21.15.77:0/3681385175 >> 172.21.15.77:0/3681385175 conn(0x5629ac8ab210 msgr2=0x5629ac9724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 -- 172.21.15.77:0/3681385175 shutdown_connections 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 -- 172.21.15.77:0/3681385175 wait complete. 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.536+0000 7f3b5038fb80 1 -- start start 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b5038fb80 1 Processor -- start 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac97d970 0x5629ac98fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97e330 0x5629ac98a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b5038fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac983340 0x5629ac98ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401af0d0 con 0x5629ac97e330 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401aea00 con 0x5629ac97d970 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b5038fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3b401abc00 con 0x5629ac983340 2024-05-27T08:43:03.541 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b5038fb80 10 client.0 _ll_register_callbacks cb 0x5629ac867ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97e330 0x5629ac98a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac97d970 0x5629ac98fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b3effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97e330 0x5629ac98a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:57380/0 (socket says 172.21.15.77:57380) 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b3f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac97d970 0x5629ac98fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:44110/0 (socket says 172.21.15.77:44110) 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b3effd700 1 -- 172.21.15.77:0/2700401617 learned_addr learned my addr 172.21.15.77:0/2700401617 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.537+0000 7f3b3ffff700 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac983340 0x5629ac98ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/2700401617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac983340 msgr2=0x5629ac98ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac983340 0x5629ac98ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/2700401617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97e330 msgr2=0x5629ac98a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97e330 0x5629ac98a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b3f7fe700 1 -- 172.21.15.77:0/2700401617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629ac98b380 con 0x5629ac97d970 2024-05-27T08:43:03.542 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b3f7fe700 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac97d970 0x5629ac98fd90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3b300a99e0 tx=0x7f3b300a9f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9feb40379bdfa7c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b1effd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b1effd700 1 -- 172.21.15.77:0/2700401617 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300aa150 con 0x5629ac97d970 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b1effd700 1 -- 172.21.15.77:0/2700401617 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f3b30010720 con 0x5629ac97d970 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2700401617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b300b2040 con 0x5629ac97d970 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b1effd700 1 -- 172.21.15.77:0/2700401617 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3b300ade00 con 0x5629ac97d970 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b5038fb80 10 client.25115 fetch_fsmap learned FSMap version 35 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2700401617 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5629aca071e0 con 0x5629ac97d970 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.538+0000 7f3b1effd700 1 -- 172.21.15.77:0/2700401617 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b300b2040 con 0x5629ac97d970 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.539+0000 7f3b1effd700 1 -- 172.21.15.77:0/2700401617 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f3b300aef90 con 0x5629ac97d970 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.539+0000 7f3b5038fb80 10 client.25115 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.539+0000 7f3b5038fb80 10 client.25115 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.539+0000 7f3b5038fb80 20 client.25115 populate_metadata read hostname 'smithi077' 2024-05-27T08:43:03.543 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.539+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5629aca07690 0x5629aca09ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.539+0000 7f3b5038fb80 4 client.25115 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-05-27T08:43:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.539+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2700401617 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x5629ac8ded70 con 0x5629aca07690 2024-05-27T08:43:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.539+0000 7f3b3effd700 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5629aca07690 0x5629aca09ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.540+0000 7f3b3effd700 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5629aca07690 0x5629aca09ab0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f8e5ff3da42f0bc server_cookie=e6faf2dbaed44503 in_seq=0 out_seq=0 2024-05-27T08:43:03.544 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.540+0000 7f3b1effd700 10 client.25115 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:03.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125854 3.44 MB/sec execute 57 sec latency 913.473 ms 2024-05-27T08:43:03.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b1effd700 1 -- 172.21.15.77:0/2700401617 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5629ac8ded70 con 0x5629aca07690 2024-05-27T08:43:03.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b1effd700 10 client.25115 handle_command_reply: tid=0 2024-05-27T08:43:03.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 client.25115 shutdown 2024-05-27T08:43:03.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2700401617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac97d970 msgr2=0x5629ac98fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac97d970 0x5629ac98fd90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3b300a99e0 tx=0x7f3b300a9f90 comp rx=0 tx=0).stop 2024-05-27T08:43:03.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2700401617 shutdown_connections 2024-05-27T08:43:03.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x5629aca07690 0x5629aca09ab0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629ac983340 0x5629ac98ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x5629ac97d970 0x5629ac98fd90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b1effd700 0 client.25115 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:03.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 --2- 172.21.15.77:0/2700401617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5629ac97e330 0x5629ac98a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.628+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2700401617 >> 172.21.15.77:0/2700401617 conn(0x5629ac8ab210 msgr2=0x5629ac972d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:03.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2700401617 shutdown_connections 2024-05-27T08:43:03.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b5038fb80 1 -- 172.21.15.77:0/2700401617 wait complete. 2024-05-27T08:43:03.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b5038fb80 20 client.25115 trim_cache size 0 max 16384 2024-05-27T08:43:03.632 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:43:03.632 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-05-27T08:43:03.632 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:43:03.632 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 14, 2024-05-27T08:43:03.632 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: "3f59cfe1": { 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 14, 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:43:03.633 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 144.5, 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:03.634 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:03.635 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:03.635 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:03.635 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:43:03.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 -- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3b280b2aa0 msgr2=0x7f3b280b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3b280b2aa0 0x7f3b280b4f60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b34000c00 tx=0x7f3b34005fb0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 -- 172.21.15.77:0/3510994555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4006beb0 msgr2=0x7f3b4019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:03.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4006beb0 0x7f3b4019f790 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000e9a0 tx=0x7f3b3000ecb0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 -- 172.21.15.77:0/3510994555 shutdown_connections 2024-05-27T08:43:03.636 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f3b280b2aa0 0x7f3b280b4f60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3b40108860 0x7f3b40199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b4006beb0 0x7f3b4019f790 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 --2- 172.21.15.77:0/3510994555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3b4006c860 0x7f3b4019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:03.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.629+0000 7f3b45af9700 1 -- 172.21.15.77:0/3510994555 >> 172.21.15.77:0/3510994555 conn(0x7f3b40078490 msgr2=0x7f3b401015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:03.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.630+0000 7f3b45af9700 1 -- 172.21.15.77:0/3510994555 shutdown_connections 2024-05-27T08:43:03.637 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:03.630+0000 7f3b45af9700 1 -- 172.21.15.77:0/3510994555 wait complete. 2024-05-27T08:43:03.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:03 smithi179 ceph-mon[36481]: pgmap v146: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2024-05-27T08:43:03.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:03 smithi077 ceph-mon[32007]: pgmap v146: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2024-05-27T08:43:03.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:03 smithi092 ceph-mon[36049]: pgmap v146: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 229 op/s 2024-05-27T08:43:04.419 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3f59cfe1', quiesce took 6.0 seconds. Will release after: 18.5 2024-05-27T08:43:04.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 3.38 MB/sec execute 58 sec latency 998.574 ms 2024-05-27T08:43:05.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 3.32 MB/sec execute 59 sec latency 1998.687 ms 2024-05-27T08:43:05.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:05 smithi179 ceph-mon[36481]: pgmap v147: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-05-27T08:43:05.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:05 smithi077 ceph-mon[32007]: pgmap v147: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-05-27T08:43:05.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:05 smithi092 ceph-mon[36049]: pgmap v147: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-05-27T08:43:06.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 3.27 MB/sec execute 60 sec latency 2998.746 ms 2024-05-27T08:43:07.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 3.21 MB/sec execute 61 sec latency 3998.809 ms 2024-05-27T08:43:07.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:07 smithi179 ceph-mon[36481]: pgmap v148: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-05-27T08:43:07.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:07 smithi077 ceph-mon[32007]: pgmap v148: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-05-27T08:43:07.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:07 smithi092 ceph-mon[36049]: pgmap v148: 97 pgs: 97 active+clean; 80 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-05-27T08:43:08.598 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 3.16 MB/sec execute 62 sec latency 4998.906 ms 2024-05-27T08:43:09.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 3.11 MB/sec execute 63 sec latency 5999.013 ms 2024-05-27T08:43:09.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:09 smithi179 ceph-mon[36481]: pgmap v149: 97 pgs: 97 active+clean; 81 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 200 op/s 2024-05-27T08:43:09.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:09 smithi077 ceph-mon[32007]: pgmap v149: 97 pgs: 97 active+clean; 81 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 200 op/s 2024-05-27T08:43:09.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:09 smithi092 ceph-mon[36049]: pgmap v149: 97 pgs: 97 active+clean; 81 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 200 op/s 2024-05-27T08:43:10.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 3.06 MB/sec execute 64 sec latency 6999.102 ms 2024-05-27T08:43:11.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 3.02 MB/sec execute 65 sec latency 7999.214 ms 2024-05-27T08:43:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:11 smithi179 ceph-mon[36481]: pgmap v150: 97 pgs: 97 active+clean; 81 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 472 KiB/s wr, 104 op/s 2024-05-27T08:43:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:11 smithi077 ceph-mon[32007]: pgmap v150: 97 pgs: 97 active+clean; 81 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 472 KiB/s wr, 104 op/s 2024-05-27T08:43:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:11 smithi092 ceph-mon[36049]: pgmap v150: 97 pgs: 97 active+clean; 81 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 472 KiB/s wr, 104 op/s 2024-05-27T08:43:12.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.97 MB/sec execute 66 sec latency 8999.271 ms 2024-05-27T08:43:13.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.93 MB/sec execute 67 sec latency 9999.365 ms 2024-05-27T08:43:13.856 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:13 smithi179 ceph-mon[36481]: pgmap v151: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 723 KiB/s wr, 116 op/s 2024-05-27T08:43:13.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:13 smithi077 ceph-mon[32007]: pgmap v151: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 723 KiB/s wr, 116 op/s 2024-05-27T08:43:13.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:13 smithi092 ceph-mon[36049]: pgmap v151: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 723 KiB/s wr, 116 op/s 2024-05-27T08:43:14.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.88 MB/sec execute 68 sec latency 10999.456 ms 2024-05-27T08:43:15.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.84 MB/sec execute 69 sec latency 11999.507 ms 2024-05-27T08:43:15.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:15 smithi179 ceph-mon[36481]: pgmap v152: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 348 KiB/s wr, 36 op/s 2024-05-27T08:43:15.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:15 smithi077 ceph-mon[32007]: pgmap v152: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 348 KiB/s wr, 36 op/s 2024-05-27T08:43:15.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:15 smithi092 ceph-mon[36049]: pgmap v152: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 348 KiB/s wr, 36 op/s 2024-05-27T08:43:16.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.80 MB/sec execute 70 sec latency 12999.571 ms 2024-05-27T08:43:17.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.76 MB/sec execute 71 sec latency 13999.610 ms 2024-05-27T08:43:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:17 smithi179 ceph-mon[36481]: pgmap v153: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 348 KiB/s wr, 36 op/s 2024-05-27T08:43:17.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:17 smithi077 ceph-mon[32007]: pgmap v153: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 348 KiB/s wr, 36 op/s 2024-05-27T08:43:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:17 smithi092 ceph-mon[36049]: pgmap v153: 97 pgs: 97 active+clean; 84 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 348 KiB/s wr, 36 op/s 2024-05-27T08:43:18.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.72 MB/sec execute 72 sec latency 14999.645 ms 2024-05-27T08:43:19.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.69 MB/sec execute 73 sec latency 15999.693 ms 2024-05-27T08:43:19.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:19 smithi179 ceph-mon[36481]: pgmap v154: 97 pgs: 97 active+clean; 84 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 610 KiB/s wr, 54 op/s 2024-05-27T08:43:19.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:19 smithi077 ceph-mon[32007]: pgmap v154: 97 pgs: 97 active+clean; 84 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 610 KiB/s wr, 54 op/s 2024-05-27T08:43:19.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:19 smithi092 ceph-mon[36049]: pgmap v154: 97 pgs: 97 active+clean; 84 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 610 KiB/s wr, 54 op/s 2024-05-27T08:43:20.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.65 MB/sec execute 74 sec latency 16999.758 ms 2024-05-27T08:43:21.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.61 MB/sec execute 75 sec latency 17999.803 ms 2024-05-27T08:43:21.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:21 smithi179 ceph-mon[36481]: pgmap v155: 97 pgs: 97 active+clean; 84 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 513 KiB/s wr, 30 op/s 2024-05-27T08:43:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:21 smithi077 ceph-mon[32007]: pgmap v155: 97 pgs: 97 active+clean; 84 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 513 KiB/s wr, 30 op/s 2024-05-27T08:43:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:21 smithi092 ceph-mon[36049]: pgmap v155: 97 pgs: 97 active+clean; 84 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 513 KiB/s wr, 30 op/s 2024-05-27T08:43:22.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.58 MB/sec execute 76 sec latency 18999.860 ms 2024-05-27T08:43:22.920 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:43:23.179 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:43:23.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.55 MB/sec execute 77 sec latency 19999.901 ms 2024-05-27T08:43:23.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:23 smithi077 ceph-mon[32007]: pgmap v156: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 777 KiB/s wr, 47 op/s 2024-05-27T08:43:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:23 smithi092 ceph-mon[36049]: pgmap v156: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 777 KiB/s wr, 47 op/s 2024-05-27T08:43:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:23 smithi179 ceph-mon[36481]: pgmap v156: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 777 KiB/s wr, 47 op/s 2024-05-27T08:43:24.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.51 MB/sec execute 78 sec latency 20999.946 ms 2024-05-27T08:43:24.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.879+0000 7f9c7e815700 1 -- 172.21.15.77:0/4251704367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806b220 msgr2=0x7f9c7806b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.879+0000 7f9c7e815700 1 --2- 172.21.15.77:0/4251704367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806b220 0x7f9c7806b6a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009990 tx=0x7f9c60009ca0 comp rx=0 tx=0).stop 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.879+0000 7f9c7e815700 1 -- 172.21.15.77:0/4251704367 shutdown_connections 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.879+0000 7f9c7e815700 1 --2- 172.21.15.77:0/4251704367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7806bbe0 0x7f9c781103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.879+0000 7f9c7e815700 1 --2- 172.21.15.77:0/4251704367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c781095b0 0x7f9c78109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.879+0000 7f9c7e815700 1 --2- 172.21.15.77:0/4251704367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806b220 0x7f9c7806b6a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.879+0000 7f9c7e815700 1 -- 172.21.15.77:0/4251704367 >> 172.21.15.77:0/4251704367 conn(0x7f9c780793f0 msgr2=0x7f9c78079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 -- 172.21.15.77:0/4251704367 shutdown_connections 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 -- 172.21.15.77:0/4251704367 wait complete. 2024-05-27T08:43:24.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 Processor -- start 2024-05-27T08:43:24.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 -- start start 2024-05-27T08:43:24.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7806b220 0x7f9c7819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:24.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806bbe0 0x7f9c7819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:24.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c781095b0 0x7f9c78199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:24.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c781136c0 con 0x7f9c7806bbe0 2024-05-27T08:43:24.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78113540 con 0x7f9c7806b220 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c7e815700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9c78113840 con 0x7f9c781095b0 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.880+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806bbe0 0x7f9c7819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c6ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806bbe0 0x7f9c7819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46638/0 (socket says 172.21.15.77:46638) 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c6ffff700 1 -- 172.21.15.77:0/2228660535 learned_addr learned my addr 172.21.15.77:0/2228660535 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c77fff700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7806b220 0x7f9c7819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c7cdb2700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c781095b0 0x7f9c78199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c6ffff700 1 -- 172.21.15.77:0/2228660535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c781095b0 msgr2=0x7f9c78199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c6ffff700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c781095b0 0x7f9c78199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c6ffff700 1 -- 172.21.15.77:0/2228660535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7806b220 msgr2=0x7f9c7819f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c6ffff700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7806b220 0x7f9c7819f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c6ffff700 1 -- 172.21.15.77:0/2228660535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7819a1a0 con 0x7f9c7806bbe0 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c6ffff700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806bbe0 0x7f9c7819fdc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9c60000c00 tx=0x7f9c600047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c75ffb700 1 -- 172.21.15.77:0/2228660535 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60027070 con 0x7f9c7806bbe0 2024-05-27T08:43:24.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c75ffb700 1 -- 172.21.15.77:0/2228660535 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c600040e0 con 0x7f9c7806bbe0 2024-05-27T08:43:24.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c75ffb700 1 -- 172.21.15.77:0/2228660535 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000f460 con 0x7f9c7806bbe0 2024-05-27T08:43:24.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7819a430 con 0x7f9c7806bbe0 2024-05-27T08:43:24.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.881+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c781a6750 con 0x7f9c7806bbe0 2024-05-27T08:43:24.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.882+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c78104c70 con 0x7f9c7806bbe0 2024-05-27T08:43:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.884+0000 7f9c75ffb700 1 -- 172.21.15.77:0/2228660535 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9c6002a070 con 0x7f9c7806bbe0 2024-05-27T08:43:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.884+0000 7f9c75ffb700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c640b29f0 0x7f9c640b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:24.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.885+0000 7f9c77fff700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c640b29f0 0x7f9c640b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.885+0000 7f9c75ffb700 1 -- 172.21.15.77:0/2228660535 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f9c600e3a90 con 0x7f9c7806bbe0 2024-05-27T08:43:24.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.885+0000 7f9c77fff700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c640b29f0 0x7f9c640b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c68005950 tx=0x7f9c6800a300 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:24.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:24.888+0000 7f9c75ffb700 1 -- 172.21.15.77:0/2228660535 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c60031070 con 0x7f9c7806bbe0 2024-05-27T08:43:25.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.144+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c7802e560 con 0x7f9c7806bbe0 2024-05-27T08:43:25.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.144+0000 7f9c75ffb700 1 -- 172.21.15.77:0/2228660535 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11927 (secure 0 0 0) 0x7f9c60013070 con 0x7f9c7806bbe0 2024-05-27T08:43:25.146 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:43:25.146 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:42:58.669173+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24418,24434,24410,24412,24433]},"id":1}]} 2024-05-27T08:43:25.148 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c640b29f0 msgr2=0x7f9c640b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c640b29f0 0x7f9c640b4eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c68005950 tx=0x7f9c6800a300 comp rx=0 tx=0).stop 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806bbe0 msgr2=0x7f9c7819fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806bbe0 0x7f9c7819fdc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9c60000c00 tx=0x7f9c600047b0 comp rx=0 tx=0).stop 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 shutdown_connections 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c640b29f0 0x7f9c640b4eb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c781095b0 0x7f9c78199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c7806b220 0x7f9c7819f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 --2- 172.21.15.77:0/2228660535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c7806bbe0 0x7f9c7819fdc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 >> 172.21.15.77:0/2228660535 conn(0x7f9c780793f0 msgr2=0x7f9c781011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 shutdown_connections 2024-05-27T08:43:25.149 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:25.148+0000 7f9c7e815700 1 -- 172.21.15.77:0/2228660535 wait complete. 2024-05-27T08:43:25.150 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-05-27T08:43:25.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.48 MB/sec execute 79 sec latency 21999.971 ms 2024-05-27T08:43:25.858 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:25 smithi077 ceph-mon[32007]: pgmap v157: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 526 KiB/s wr, 35 op/s 2024-05-27T08:43:25.858 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:25 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2228660535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:25.889 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 3f59cfe1 --release --await' 2024-05-27T08:43:25.889 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db --set-id 3f59cfe1 --release --await 2024-05-27T08:43:25.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:25 smithi092 ceph-mon[36049]: pgmap v157: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 526 KiB/s wr, 35 op/s 2024-05-27T08:43:25.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:25 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2228660535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:25 smithi179 ceph-mon[36481]: pgmap v157: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 526 KiB/s wr, 35 op/s 2024-05-27T08:43:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:25 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2228660535' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:26.138 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:43:26.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.45 MB/sec execute 80 sec latency 23000.000 ms 2024-05-27T08:43:26.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:26 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:43:26.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:26 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:43:27.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:26 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:43:27.599 INFO:tasks.workunit.client.0.smithi077.stdout: 1 125855 2.42 MB/sec execute 81 sec latency 24000.029 ms 2024-05-27T08:43:27.847 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 -- 172.21.15.77:0/2290631168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f085406c650 msgr2=0x7f08541103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 --2- 172.21.15.77:0/2290631168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f085406c650 0x7f08541103d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b0a0 tx=0x7f084c00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 -- 172.21.15.77:0/2290631168 shutdown_connections 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 --2- 172.21.15.77:0/2290631168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f085406c650 0x7f08541103d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 --2- 172.21.15.77:0/2290631168 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f08541095b0 0x7f0854109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 --2- 172.21.15.77:0/2290631168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f085406bc90 0x7f085406c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 -- 172.21.15.77:0/2290631168 >> 172.21.15.77:0/2290631168 conn(0x7f0854079410 msgr2=0x7f0854079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 -- 172.21.15.77:0/2290631168 shutdown_connections 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.846+0000 7f085b998700 1 -- 172.21.15.77:0/2290631168 wait complete. 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f085b998700 1 Processor -- start 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f085b998700 1 -- start start 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f085b998700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f085406bc90 0x7f085419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.848 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f085b998700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f085406c650 0x7f085419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f085b998700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08541095b0 0x7f08541999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f085b998700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08541136f0 con 0x7f08541095b0 2024-05-27T08:43:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f085b998700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f0854113570 con 0x7f085406c650 2024-05-27T08:43:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f085b998700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0854113870 con 0x7f085406bc90 2024-05-27T08:43:27.849 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0858f33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f085406c650 0x7f085419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.849 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:27 smithi077 ceph-mon[32007]: pgmap v158: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 526 KiB/s wr, 35 op/s 2024-05-27T08:43:27.849 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:27 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:43:27.849 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:27 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:43:27.849 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:27 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:43:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0858f33700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f085406c650 0x7f085419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:42892/0 (socket says 172.21.15.77:42892) 2024-05-27T08:43:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0858f33700 1 -- 172.21.15.77:0/414450069 learned_addr learned my addr 172.21.15.77:0/414450069 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:27.850 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0859f35700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08541095b0 0x7f08541999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0859734700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f085406bc90 0x7f085419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0859f35700 1 -- 172.21.15.77:0/414450069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f085406bc90 msgr2=0x7f085419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0859f35700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f085406bc90 0x7f085419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0859f35700 1 -- 172.21.15.77:0/414450069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f085406c650 msgr2=0x7f085419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0859f35700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f085406c650 0x7f085419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0859f35700 1 -- 172.21.15.77:0/414450069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085419a150 con 0x7f08541095b0 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.847+0000 7f0858f33700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f085406c650 0x7f085419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.848+0000 7f0859f35700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08541095b0 0x7f08541999f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b070 tx=0x7f084c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.848+0000 7f08467fc700 1 -- 172.21.15.77:0/414450069 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c022070 con 0x7f08541095b0 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.848+0000 7f085b998700 1 -- 172.21.15.77:0/414450069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085419a3e0 con 0x7f08541095b0 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.848+0000 7f08467fc700 1 -- 172.21.15.77:0/414450069 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f084c007b80 con 0x7f08541095b0 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.848+0000 7f08467fc700 1 -- 172.21.15.77:0/414450069 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0052a0 con 0x7f08541095b0 2024-05-27T08:43:27.851 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.848+0000 7f085b998700 1 -- 172.21.15.77:0/414450069 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0854102470 con 0x7f08541095b0 2024-05-27T08:43:27.852 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.851+0000 7f08467fc700 1 -- 172.21.15.77:0/414450069 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f084c02d020 con 0x7f08541095b0 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f086622eb80 1 Processor -- start 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f086622eb80 1 -- start start 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f08467fc700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f08400b29f0 0x7f08400b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f086622eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fb12b0 0x55bfd8fb1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f086622eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fabd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f0858f33700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f08400b29f0 0x7f08400b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f086622eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f086622eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bfd8f0ad90 con 0x55bfd8fab8e0 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f086622eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bfd8efa930 con 0x55bfd8fac2a0 2024-05-27T08:43:27.854 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f086622eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bfd8efe970 con 0x55bfd8fb12b0 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.852+0000 7f08467fc700 1 -- 172.21.15.77:0/414450069 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f084c00f020 con 0x7f08541095b0 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fabd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fabd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46662/0 (socket says 172.21.15.77:46662) 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 -- 172.21.15.77:0/1213986437 learned_addr learned my addr 172.21.15.77:0/1213986437 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0859734700 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fb12b0 0x55bfd8fb1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0859f35700 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f08400b29f0 0x7f08400b4eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08480139f0 tx=0x7f0848013db0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 -- 172.21.15.77:0/1213986437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fb12b0 msgr2=0x55bfd8fb1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fb12b0 0x55bfd8fb1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 -- 172.21.15.77:0/1213986437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fac2a0 msgr2=0x55bfd8fb80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.855 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 -- 172.21.15.77:0/1213986437 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfd8e4a9d0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0859f35700 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:43:27.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f0858f33700 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fabd60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0848011ba0 tx=0x7f084801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665a91ec6edbe519 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:27.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f083f7fe700 1 -- 172.21.15.77:0/1213986437 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0848004a30 con 0x55bfd8fab8e0 2024-05-27T08:43:27.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f083f7fe700 1 -- 172.21.15.77:0/1213986437 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08480054d0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.853+0000 7f083f7fe700 1 -- 172.21.15.77:0/1213986437 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084801e9a0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 -- 172.21.15.77:0/1213986437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 msgr2=0x55bfd8fabd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fabd60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0848011ba0 tx=0x7f084801e000 comp rx=0 tx=0).stop 2024-05-27T08:43:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 -- 172.21.15.77:0/1213986437 shutdown_connections 2024-05-27T08:43:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fb12b0 0x55bfd8fb1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 --2- 172.21.15.77:0/1213986437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fabd60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 -- 172.21.15.77:0/1213986437 >> 172.21.15.77:0/1213986437 conn(0x55bfd8ed9180 msgr2=0x55bfd8fa61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:27.860 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 -- 172.21.15.77:0/1213986437 shutdown_connections 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 -- 172.21.15.77:0/1213986437 wait complete. 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.854+0000 7f086622eb80 1 -- start start 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f086622eb80 1 Processor -- start 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f086622eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f086622eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f086622eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fb12b0 0x55bfd8fb55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f086622eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bfd8e010a0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f086622eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55bfd8e9c8c0 con 0x55bfd8fb12b0 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f086622eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bfd8fb8680 con 0x55bfd8fac2a0 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f086622eb80 10 client.0 _ll_register_callbacks cb 0x55bfd8e95a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859734700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859734700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46672/0 (socket says 172.21.15.77:46672) 2024-05-27T08:43:27.861 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859734700 1 -- 172.21.15.77:0/625585398 learned_addr learned my addr 172.21.15.77:0/625585398 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859f35700 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fb12b0 0x55bfd8fb55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0858f33700 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859734700 1 -- 172.21.15.77:0/625585398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 msgr2=0x55bfd8fb50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859734700 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859734700 1 -- 172.21.15.77:0/625585398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fb12b0 msgr2=0x55bfd8fb55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859734700 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fb12b0 0x55bfd8fb55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.855+0000 7f0859734700 1 -- 172.21.15.77:0/625585398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfd8e99790 con 0x55bfd8fab8e0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f0859734700 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb7db0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f085000bfd0 tx=0x7f0850011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135f158ac182b0b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f083cff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f083cff9700 1 -- 172.21.15.77:0/625585398 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850004030 con 0x55bfd8fab8e0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f083cff9700 1 -- 172.21.15.77:0/625585398 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f08500041d0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f086622eb80 1 -- 172.21.15.77:0/625585398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08480054d0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f083cff9700 1 -- 172.21.15.77:0/625585398 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f08500043d0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f086622eb80 10 client.15621 fetch_fsmap learned FSMap version 35 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f086622eb80 1 -- 172.21.15.77:0/625585398 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bfd8fc2e80 con 0x55bfd8fab8e0 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f083cff9700 1 -- 172.21.15.77:0/625585398 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0850004ce0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f083cff9700 1 -- 172.21.15.77:0/625585398 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f0850010700 con 0x55bfd8fab8e0 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f086622eb80 10 client.15621 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f086622eb80 10 client.15621 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.856+0000 7f086622eb80 20 client.15621 populate_metadata read hostname 'smithi077' 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.857+0000 7f086622eb80 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55bfd8fc5510 0x55bfd8fc7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.857+0000 7f086622eb80 4 client.15621 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:43:27.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.857+0000 7f086622eb80 1 -- 172.21.15.77:0/625585398 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f08480054d0 con 0x55bfd8fc5510 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.857+0000 7f0858f33700 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55bfd8fc5510 0x55bfd8fc7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.857+0000 7f083cff9700 10 client.15621 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.857+0000 7f0858f33700 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55bfd8fc5510 0x55bfd8fc7950 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0214b9a9379756 server_cookie=1345b5923e10c945 in_seq=0 out_seq=0 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f083cff9700 1 -- 172.21.15.77:0/625585398 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f08480054d0 con 0x55bfd8fc5510 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f083cff9700 10 client.15621 handle_command_reply: tid=0 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 client.15621 shutdown 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 -- 172.21.15.77:0/625585398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 msgr2=0x55bfd8fb7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb7db0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f085000bfd0 tx=0x7f0850011720 comp rx=0 tx=0).stop 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 -- 172.21.15.77:0/625585398 shutdown_connections 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55bfd8fc5510 0x55bfd8fc7950 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fb12b0 0x55bfd8fb55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb7db0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 --2- 172.21.15.77:0/625585398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 -- 172.21.15.77:0/625585398 >> 172.21.15.77:0/625585398 conn(0x55bfd8ed9180 msgr2=0x55bfd8fa0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:27.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f083cff9700 0 client.15621 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:27.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 -- 172.21.15.77:0/625585398 shutdown_connections 2024-05-27T08:43:27.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 1 -- 172.21.15.77:0/625585398 wait complete. 2024-05-27T08:43:27.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.858+0000 7f086622eb80 20 client.15621 trim_cache size 0 max 16384 2024-05-27T08:43:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f086622eb80 1 Processor -- start 2024-05-27T08:43:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f086622eb80 1 -- start start 2024-05-27T08:43:27.879 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f086622eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8ed9180 0x55bfd8fa0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f086622eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fa0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f086622eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fa0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f086622eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08541abab0 con 0x55bfd8ed9180 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f086622eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f08541ad5f0 con 0x55bfd8fab8e0 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f086622eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f08541ac180 con 0x55bfd8fac2a0 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f0859734700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8ed9180 0x55bfd8fa0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f0859734700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8ed9180 0x55bfd8fa0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46684/0 (socket says 172.21.15.77:46684) 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.878+0000 7f0859734700 1 -- 172.21.15.77:0/2992365797 learned_addr learned my addr 172.21.15.77:0/2992365797 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f0859734700 1 -- 172.21.15.77:0/2992365797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 msgr2=0x55bfd8fa0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f0859734700 1 --2- 172.21.15.77:0/2992365797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fa0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f0859734700 1 -- 172.21.15.77:0/2992365797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 msgr2=0x55bfd8fa0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f0859734700 1 --2- 172.21.15.77:0/2992365797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fa0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f0859734700 1 -- 172.21.15.77:0/2992365797 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfd8fb46f0 con 0x55bfd8ed9180 2024-05-27T08:43:27.880 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f0859734700 1 --2- 172.21.15.77:0/2992365797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8ed9180 0x55bfd8fa0410 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0850001eb0 tx=0x7f0850011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22247642870edd0b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:27.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f083effd700 1 -- 172.21.15.77:0/2992365797 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850000e60 con 0x55bfd8ed9180 2024-05-27T08:43:27.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f083effd700 1 -- 172.21.15.77:0/2992365797 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08500054d0 con 0x55bfd8ed9180 2024-05-27T08:43:27.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.879+0000 7f083effd700 1 -- 172.21.15.77:0/2992365797 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850004aa0 con 0x55bfd8ed9180 2024-05-27T08:43:27.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- 172.21.15.77:0/2992365797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8ed9180 msgr2=0x55bfd8fa0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 --2- 172.21.15.77:0/2992365797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8ed9180 0x55bfd8fa0410 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0850001eb0 tx=0x7f0850011b90 comp rx=0 tx=0).stop 2024-05-27T08:43:27.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- 172.21.15.77:0/2992365797 shutdown_connections 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 --2- 172.21.15.77:0/2992365797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fa0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 --2- 172.21.15.77:0/2992365797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8ed9180 0x55bfd8fa0410 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 --2- 172.21.15.77:0/2992365797 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fa0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- 172.21.15.77:0/2992365797 >> 172.21.15.77:0/2992365797 conn(0x55bfd8fc7800 msgr2=0x55bfd8fab160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- 172.21.15.77:0/2992365797 shutdown_connections 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- 172.21.15.77:0/2992365797 wait complete. 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- start start 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 Processor -- start 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8ed9180 0x55bfd8fb88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f08541acf20 con 0x55bfd8fac2a0 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f08541ac850 con 0x55bfd8fab8e0 2024-05-27T08:43:27.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.880+0000 7f086622eb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f08541a9a90 con 0x55bfd8ed9180 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f086622eb80 10 client.0 _ll_register_callbacks cb 0x55bfd8e95a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:46692/0 (socket says 172.21.15.77:46692) 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 -- 172.21.15.77:0/569047324 learned_addr learned my addr 172.21.15.77:0/569047324 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859734700 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8ed9180 0x55bfd8fb88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0858f33700 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 -- 172.21.15.77:0/569047324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8ed9180 msgr2=0x55bfd8fb88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8ed9180 0x55bfd8fb88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 -- 172.21.15.77:0/569047324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 msgr2=0x55bfd8fb8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 -- 172.21.15.77:0/569047324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfd90302f0 con 0x55bfd8fac2a0 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f0859f35700 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb9350 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f084c0aa5f0 tx=0x7f084c0ae080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5f3f9d6f17b683 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f083cff9700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:43:27.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f083cff9700 1 -- 172.21.15.77:0/569047324 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c0c8060 con 0x55bfd8fac2a0 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f083cff9700 1 -- 172.21.15.77:0/569047324 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f084c0ae7d0 con 0x55bfd8fac2a0 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f086622eb80 1 -- 172.21.15.77:0/569047324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08500054d0 con 0x55bfd8fac2a0 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f083cff9700 1 -- 172.21.15.77:0/569047324 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f084c0b2920 con 0x55bfd8fac2a0 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f086622eb80 10 client.15627 fetch_fsmap learned FSMap version 35 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.881+0000 7f086622eb80 1 -- 172.21.15.77:0/569047324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bfd9031670 con 0x55bfd8fac2a0 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f083cff9700 1 -- 172.21.15.77:0/569047324 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08500054d0 con 0x55bfd8fac2a0 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f083cff9700 1 -- 172.21.15.77:0/569047324 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f084c011070 con 0x55bfd8fac2a0 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f086622eb80 10 client.15627 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f086622eb80 10 client.15627 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f086622eb80 20 client.15627 populate_metadata read hostname 'smithi077' 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f086622eb80 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55bfd8fb12b0 0x55bfd9033d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f086622eb80 4 client.15627 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3f59cfe1", "release": true, "await": true}] 2024-05-27T08:43:27.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f086622eb80 1 -- 172.21.15.77:0/569047324 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3f59cfe1", "release": true, "await": true}) -- 0x55bfd8f77ae0 con 0x55bfd8fb12b0 2024-05-27T08:43:27.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f0859734700 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55bfd8fb12b0 0x55bfd9033d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:27.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f0859734700 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55bfd8fb12b0 0x55bfd9033d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=246a7024fe280259 server_cookie=e6ba9d72308697c2 in_seq=0 out_seq=0 2024-05-27T08:43:27.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.882+0000 7f083cff9700 10 client.15627 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:27.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.893+0000 7f083cff9700 1 -- 172.21.15.77:0/569047324 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bfd8f77ae0 con 0x55bfd8fb12b0 2024-05-27T08:43:27.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.893+0000 7f083cff9700 10 client.15627 handle_command_reply: tid=0 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.893+0000 7f086622eb80 1 client.15627 shutdown 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 -- 172.21.15.77:0/569047324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fac2a0 msgr2=0x55bfd8fb9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb9350 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f084c0aa5f0 tx=0x7f084c0ae080 comp rx=0 tx=0).stop 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 -- 172.21.15.77:0/569047324 shutdown_connections 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55bfd8fb12b0 0x55bfd9033d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bfd8fac2a0 0x55bfd8fb9350 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f083cff9700 0 client.15627 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfd8ed9180 0x55bfd8fb88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 --2- 172.21.15.77:0/569047324 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55bfd8fab8e0 0x55bfd8fb8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 -- 172.21.15.77:0/569047324 >> 172.21.15.77:0/569047324 conn(0x55bfd8fc7800 msgr2=0x55bfd8fb4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 -- 172.21.15.77:0/569047324 shutdown_connections 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 1 -- 172.21.15.77:0/569047324 wait complete. 2024-05-27T08:43:27.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.894+0000 7f086622eb80 20 client.15627 trim_cache size 0 max 16384 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 16, 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "3f59cfe1": { 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 16, 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:43:27.896 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 144.5, 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:27.897 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:27.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:27.898 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:27.898 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:43:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 -- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f08400b29f0 msgr2=0x7f08400b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f08400b29f0 0x7f08400b4eb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08480139f0 tx=0x7f0848013db0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 -- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08541095b0 msgr2=0x7f08541999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08541095b0 0x7f08541999f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f084c00b070 tx=0x7f084c004ce0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 -- 172.21.15.77:0/414450069 shutdown_connections 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f08400b29f0 0x7f08400b4eb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f08541095b0 0x7f08541999f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f085406bc90 0x7f085419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 --2- 172.21.15.77:0/414450069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f085406c650 0x7f085419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 -- 172.21.15.77:0/414450069 >> 172.21.15.77:0/414450069 conn(0x7f0854079410 msgr2=0x7f08541011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 -- 172.21.15.77:0/414450069 shutdown_connections 2024-05-27T08:43:27.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:27.895+0000 7f085b998700 1 -- 172.21.15.77:0/414450069 wait complete. 2024-05-27T08:43:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:27 smithi092 ceph-mon[36049]: pgmap v158: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 526 KiB/s wr, 35 op/s 2024-05-27T08:43:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:27 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:43:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:27 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:43:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:27 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:43:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:27 smithi179 ceph-mon[36481]: pgmap v158: 97 pgs: 97 active+clean; 83 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 526 KiB/s wr, 35 op/s 2024-05-27T08:43:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:27 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:43:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:27 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:43:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:27 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:43:28.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 127541 2.50 MB/sec execute 82 sec latency 24290.804 ms 2024-05-27T08:43:28.604 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3f59cfe1', total seconds elapsed: 30.2 2024-05-27T08:43:28.604 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-05-27T08:43:28.605 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:43:28.861 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:43:29.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 128800 2.55 MB/sec execute 83 sec latency 10.595 ms 2024-05-27T08:43:29.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:29 smithi077 ceph-mon[32007]: pgmap v159: 97 pgs: 97 active+clean; 87 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-05-27T08:43:29.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:29 smithi092 ceph-mon[36049]: pgmap v159: 97 pgs: 97 active+clean; 87 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-05-27T08:43:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:29 smithi179 ceph-mon[36481]: pgmap v159: 97 pgs: 97 active+clean; 87 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.1 MiB/s wr, 56 op/s 2024-05-27T08:43:30.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 131419 2.71 MB/sec execute 84 sec latency 42.306 ms 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.647+0000 7fd2147e4700 1 -- 172.21.15.77:0/505797764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 msgr2=0x7fd20c108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.647+0000 7fd2147e4700 1 --2- 172.21.15.77:0/505797764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 0x7fd20c108000 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b0d0 tx=0x7fd20400b3e0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.647+0000 7fd2147e4700 1 -- 172.21.15.77:0/505797764 shutdown_connections 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.647+0000 7fd2147e4700 1 --2- 172.21.15.77:0/505797764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd20c102910 0x7fd20c10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.647+0000 7fd2147e4700 1 --2- 172.21.15.77:0/505797764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 0x7fd20c108000 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.647+0000 7fd2147e4700 1 --2- 172.21.15.77:0/505797764 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c101f50 0x7fd20c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 -- 172.21.15.77:0/505797764 >> 172.21.15.77:0/505797764 conn(0x7fd20c078640 msgr2=0x7fd20c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 -- 172.21.15.77:0/505797764 shutdown_connections 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 -- 172.21.15.77:0/505797764 wait complete. 2024-05-27T08:43:30.675 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 Processor -- start 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 -- start start 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c101f50 0x7fd20c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd20c102910 0x7fd20c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 0x7fd20c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c1114f0 con 0x7fd20c107c20 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c111370 con 0x7fd20c101f50 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.648+0000 7fd2147e4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c111670 con 0x7fd20c102910 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 0x7fd20c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212580700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c101f50 0x7fd20c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 0x7fd20c1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43116/0 (socket says 172.21.15.77:43116) 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 -- 172.21.15.77:0/3982020676 learned_addr learned my addr 172.21.15.77:0/3982020676 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd211d7f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd20c102910 0x7fd20c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 -- 172.21.15.77:0/3982020676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd20c102910 msgr2=0x7fd20c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:30.676 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd20c102910 0x7fd20c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 -- 172.21.15.77:0/3982020676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c101f50 msgr2=0x7fd20c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c101f50 0x7fd20c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 -- 172.21.15.77:0/3982020676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c19a140 con 0x7fd20c107c20 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212580700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c101f50 0x7fd20c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd212d81700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 0x7fd20c1999e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00db90 tx=0x7fd1fc00dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.649+0000 7fd20b7fe700 1 -- 172.21.15.77:0/3982020676 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00ac10 con 0x7fd20c107c20 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.650+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c19a430 con 0x7fd20c107c20 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.650+0000 7fd20b7fe700 1 -- 172.21.15.77:0/3982020676 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1fc00adb0 con 0x7fd20c107c20 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.650+0000 7fd20b7fe700 1 -- 172.21.15.77:0/3982020676 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc00e450 con 0x7fd20c107c20 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.650+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c1a66f0 con 0x7fd20c107c20 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.651+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd20c06a230 con 0x7fd20c107c20 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.652+0000 7fd20b7fe700 1 -- 172.21.15.77:0/3982020676 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd1fc004750 con 0x7fd20c107c20 2024-05-27T08:43:30.677 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.652+0000 7fd20b7fe700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd1f40b2a70 0x7fd1f40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:30.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.653+0000 7fd212580700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd1f40b2a70 0x7fd1f40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:30.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.653+0000 7fd20b7fe700 1 -- 172.21.15.77:0/3982020676 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fd1fc0e20b0 con 0x7fd20c107c20 2024-05-27T08:43:30.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.653+0000 7fd212580700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd1f40b2a70 0x7fd1f40b4f30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd2040095c0 tx=0x7fd20400b040 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:30.678 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.657+0000 7fd20b7fe700 1 -- 172.21.15.77:0/3982020676 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1fc0a9c50 con 0x7fd20c107c20 2024-05-27T08:43:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.928+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd20c02e560 con 0x7fd20c107c20 2024-05-27T08:43:30.929 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.929+0000 7fd20b7fe700 1 -- 172.21.15.77:0/3982020676 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11927 (secure 0 0 0) 0x7fd1fc0910e0 con 0x7fd20c107c20 2024-05-27T08:43:30.930 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:43:30.930 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:42:58.669173+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24418,24434,24410,24412,24433]},"id":1}]} 2024-05-27T08:43:30.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd1f40b2a70 msgr2=0x7fd1f40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:30.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd1f40b2a70 0x7fd1f40b4f30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd2040095c0 tx=0x7fd20400b040 comp rx=0 tx=0).stop 2024-05-27T08:43:30.932 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 msgr2=0x7fd20c1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 0x7fd20c1999e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00db90 tx=0x7fd1fc00dea0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 shutdown_connections 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd1f40b2a70 0x7fd1f40b4f30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd20c107c20 0x7fd20c1999e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd20c101f50 0x7fd20c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 --2- 172.21.15.77:0/3982020676 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd20c102910 0x7fd20c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 >> 172.21.15.77:0/3982020676 conn(0x7fd20c078640 msgr2=0x7fd20c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 shutdown_connections 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:30.932+0000 7fd2147e4700 1 -- 172.21.15.77:0/3982020676 wait complete. 2024-05-27T08:43:30.933 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-05-27T08:43:31.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 132764 2.75 MB/sec execute 85 sec latency 12.157 ms 2024-05-27T08:43:31.767 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-05-27T08:43:31.767 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-05-27T08:43:31.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:31 smithi077 ceph-mon[32007]: pgmap v160: 97 pgs: 97 active+clean; 87 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 827 KiB/s wr, 38 op/s 2024-05-27T08:43:31.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:31 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3982020676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:31 smithi092 ceph-mon[36049]: pgmap v160: 97 pgs: 97 active+clean; 87 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 827 KiB/s wr, 38 op/s 2024-05-27T08:43:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:31 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3982020676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:32.030 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:43:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:31 smithi179 ceph-mon[36481]: pgmap v160: 97 pgs: 97 active+clean; 87 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 827 KiB/s wr, 38 op/s 2024-05-27T08:43:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:31 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3982020676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:32.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 135074 2.88 MB/sec execute 86 sec latency 16.355 ms 2024-05-27T08:43:33.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137156 2.93 MB/sec execute 87 sec latency 16.333 ms 2024-05-27T08:43:33.755 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:33 smithi077 ceph-mon[32007]: pgmap v161: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-05-27T08:43:33.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 -- 172.21.15.77:0/3764309046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102470 msgr2=0x7fb2e41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:33.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 --2- 172.21.15.77:0/3764309046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102470 0x7fb2e41028f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009990 tx=0x7fb2d8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 -- 172.21.15.77:0/3764309046 shutdown_connections 2024-05-27T08:43:33.881 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 --2- 172.21.15.77:0/3764309046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e4102e30 0x7fb2e410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 --2- 172.21.15.77:0/3764309046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e41073d0 0x7fb2e41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 --2- 172.21.15.77:0/3764309046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102470 0x7fb2e41028f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 -- 172.21.15.77:0/3764309046 >> 172.21.15.77:0/3764309046 conn(0x7fb2e4079480 msgr2=0x7fb2e4079890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 -- 172.21.15.77:0/3764309046 shutdown_connections 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 -- 172.21.15.77:0/3764309046 wait complete. 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 Processor -- start 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.879+0000 7fb2ea228700 1 -- start start 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2ea228700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e4102470 0x7fb2e419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2ea228700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102e30 0x7fb2e419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2ea228700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e41073d0 0x7fb2e4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2ea228700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e4112620 con 0x7fb2e4102e30 2024-05-27T08:43:33.882 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2ea228700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41124a0 con 0x7fb2e41073d0 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2ea228700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41127a0 con 0x7fb2e4102470 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102e30 0x7fb2e419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e41073d0 0x7fb2e4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e4102470 0x7fb2e419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e41073d0 0x7fb2e4199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:44336/0 (socket says 172.21.15.77:44336) 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e4102470 0x7fb2e419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.77:44782/0 (socket says 172.21.15.77:44782) 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 -- 172.21.15.77:0/71505870 learned_addr learned my addr 172.21.15.77:0/71505870 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 -- 172.21.15.77:0/71505870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e4102470 msgr2=0x7fb2e419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e4102470 0x7fb2e419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 -- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102e30 msgr2=0x7fb2e419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102e30 0x7fb2e419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 -- 172.21.15.77:0/71505870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e419a200 con 0x7fb2e41073d0 2024-05-27T08:43:33.883 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102e30 0x7fb2e419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:43:33.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.880+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e41073d0 0x7fb2e4199aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00ef30 tx=0x7fb2dc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:33.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.881+0000 7fb2e0ff9700 1 -- 172.21.15.77:0/71505870 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0092e0 con 0x7fb2e41073d0 2024-05-27T08:43:33.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.881+0000 7fb2e0ff9700 1 -- 172.21.15.77:0/71505870 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2dc004bb0 con 0x7fb2e41073d0 2024-05-27T08:43:33.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.881+0000 7fb2ea228700 1 -- 172.21.15.77:0/71505870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e419a490 con 0x7fb2e41073d0 2024-05-27T08:43:33.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.881+0000 7fb2e0ff9700 1 -- 172.21.15.77:0/71505870 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc004020 con 0x7fb2e41073d0 2024-05-27T08:43:33.884 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.881+0000 7fb2ea228700 1 -- 172.21.15.77:0/71505870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e4103fb0 con 0x7fb2e41073d0 2024-05-27T08:43:33.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.883+0000 7fb2e0ff9700 1 -- 172.21.15.77:0/71505870 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb2dc01b030 con 0x7fb2e41073d0 2024-05-27T08:43:33.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.884+0000 7fb2e0ff9700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb2cc0b2b20 0x7fb2cc0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.884+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb2cc0b2b20 0x7fb2cc0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.884+0000 7fb2e0ff9700 1 -- 172.21.15.77:0/71505870 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb2dc0e2060 con 0x7fb2e41073d0 2024-05-27T08:43:33.885 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.884+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb2cc0b2b20 0x7fb2cc0b4fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4108920 tx=0x7fb2d400b3c0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:33.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2f4abeb80 1 Processor -- start 2024-05-27T08:43:33.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2f4abeb80 1 -- start start 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7acb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c3ff705f00 con 0x55c3ff7a8640 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c3ff6f5aa0 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c3ff6f9ae0 con 0x55c3ff7a7c80 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7acb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:44338/0 (socket says 172.21.15.77:44338) 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2e37fe700 1 -- 172.21.15.77:0/2906144158 learned_addr learned my addr 172.21.15.77:0/2906144158 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.885+0000 7fb2e37fe700 1 -- 172.21.15.77:0/2906144158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 msgr2=0x55c3ff7a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:33.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7a8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2e37fe700 1 -- 172.21.15.77:0/2906144158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 msgr2=0x55c3ff7b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2e37fe700 1 -- 172.21.15.77:0/2906144158 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3ff69f140 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7acb90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005e10 tx=0x7fb2d4013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af62a0b72ac2dafd server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2c9ffb700 1 -- 172.21.15.77:0/2906144158 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40118d0 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2c9ffb700 1 -- 172.21.15.77:0/2906144158 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d401e420 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.888 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.886+0000 7fb2c9ffb700 1 -- 172.21.15.77:0/2906144158 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d401c610 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/2906144158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 msgr2=0x55c3ff7acb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:33.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7acb90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005e10 tx=0x7fb2d4013000 comp rx=0 tx=0).stop 2024-05-27T08:43:33.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/2906144158 shutdown_connections 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7acb90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/2906144158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/2906144158 >> 172.21.15.77:0/2906144158 conn(0x55c3ff6d33d0 msgr2=0x55c3ff7a2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/2906144158 shutdown_connections 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/2906144158 wait complete. 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 -- start start 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.887+0000 7fb2f4abeb80 1 Processor -- start 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7b4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.893 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c3ff5fc130 con 0x55c3ff7a7c80 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c3ff65df30 con 0x55c3ff7a8640 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c3ff7b4d10 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2f4abeb80 10 client.0 _ll_register_callbacks cb 0x55c3ff690ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7b4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43132/0 (socket says 172.21.15.77:43132) 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e3fff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 -- 172.21.15.77:0/1109655742 learned_addr learned my addr 172.21.15.77:0/1109655742 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 -- 172.21.15.77:0/1109655742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 msgr2=0x55c3ff7b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:33.894 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 -- 172.21.15.77:0/1109655742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a8640 msgr2=0x55c3ff7b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 -- 172.21.15.77:0/1109655742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3ff645a60 con 0x55c3ff7a7c80 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7b4300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb2d401ecd0 tx=0x7fb2d401e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b5c7a1f9b60b38 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.888+0000 7fb2c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/1109655742 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4035060 con 0x55c3ff7a7c80 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/1109655742 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb2d401ea50 con 0x55c3ff7a7c80 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1109655742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d401e420 con 0x55c3ff7a7c80 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/1109655742 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb2d40118d0 con 0x55c3ff7a7c80 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 10 client.15645 fetch_fsmap learned FSMap version 35 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1109655742 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c3ff6af8f0 con 0x55c3ff7a7c80 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/1109655742 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d401e420 con 0x55c3ff7a7c80 2024-05-27T08:43:33.895 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/1109655742 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb2d4026b70 con 0x55c3ff7a7c80 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 10 client.15645 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 10 client.15645 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 20 client.15645 populate_metadata read hostname 'smithi077' 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c3ff7c16f0 0x55c3ff7c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 4 client.15645 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1109655742 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3ff707e50 con 0x55c3ff7c16f0 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.889+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c3ff7c16f0 0x55c3ff7c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.890+0000 7fb2c2ffd700 10 client.15645 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.890+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c3ff7c16f0 0x55c3ff7c3ae0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f2b97f0f16e6fb server_cookie=ee20a50989e757a4 in_seq=0 out_seq=0 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.890+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/1109655742 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c3ff707e50 con 0x55c3ff7c16f0 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.890+0000 7fb2c2ffd700 10 client.15645 handle_command_reply: tid=0 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.890+0000 7fb2f4abeb80 1 client.15645 shutdown 2024-05-27T08:43:33.896 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1109655742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a7c80 msgr2=0x55c3ff7b4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7b4300 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb2d401ecd0 tx=0x7fb2d401e5d0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1109655742 shutdown_connections 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c3ff7c16f0 0x55c3ff7c3ae0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7b4300 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2c2ffd700 0 client.15645 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1109655742 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a8640 0x55c3ff7b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1109655742 >> 172.21.15.77:0/1109655742 conn(0x55c3ff6d33d0 msgr2=0x55c3ff7a6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:33.898 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1109655742 shutdown_connections 2024-05-27T08:43:33.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1109655742 wait complete. 2024-05-27T08:43:33.899 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.891+0000 7fb2f4abeb80 20 client.15645 trim_cache size 0 max 16384 2024-05-27T08:43:33.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.911+0000 7fb2f4abeb80 1 Processor -- start 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.911+0000 7fb2f4abeb80 1 -- start start 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.911+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a7c80 0x55c3ff79cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.911+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.911+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.911+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41b0000 con 0x55c3ff7a8640 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.911+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41b1b40 con 0x55c3ff7a7c80 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.911+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41b06d0 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43140/0 (socket says 172.21.15.77:43140) 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 -- 172.21.15.77:0/1940091375 learned_addr learned my addr 172.21.15.77:0/1940091375 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:33.913 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a7c80 0x55c3ff79cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 -- 172.21.15.77:0/1940091375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 msgr2=0x55c3ff7c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 -- 172.21.15.77:0/1940091375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a7c80 msgr2=0x55c3ff79cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a7c80 0x55c3ff79cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 -- 172.21.15.77:0/1940091375 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3ff7a4080 con 0x55c3ff7a8640 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7abe50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800b2e0 tx=0x7fb2d8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3477524839ce2d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2c97fa700 1 -- 172.21.15.77:0/1940091375 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80052b0 con 0x55c3ff7a8640 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2c97fa700 1 -- 172.21.15.77:0/1940091375 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8005450 con 0x55c3ff7a8640 2024-05-27T08:43:33.914 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.912+0000 7fb2c97fa700 1 -- 172.21.15.77:0/1940091375 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800f6a0 con 0x55c3ff7a8640 2024-05-27T08:43:33.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1940091375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 msgr2=0x55c3ff7abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:33.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7abe50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800b2e0 tx=0x7fb2d8005040 comp rx=0 tx=0).stop 2024-05-27T08:43:33.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1940091375 shutdown_connections 2024-05-27T08:43:33.917 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7a7c80 0x55c3ff79cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/1940091375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7abe50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1940091375 >> 172.21.15.77:0/1940091375 conn(0x55c3ff6d33d0 msgr2=0x55c3ff7b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1940091375 shutdown_connections 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.913+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/1940091375 wait complete. 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 1 -- start start 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 1 Processor -- start 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7c7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41b1470 con 0x55c3ff7a8640 2024-05-27T08:43:33.918 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41b0da0 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e41adfa0 con 0x55c3ff7a7c80 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2f4abeb80 10 client.0 _ll_register_callbacks cb 0x55c3ff690ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2e2ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7c5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:43142/0 (socket says 172.21.15.77:43142) 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2e2ffd700 1 -- 172.21.15.77:0/393035653 learned_addr learned my addr 172.21.15.77:0/393035653 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.914+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2e3fff700 1 -- 172.21.15.77:0/393035653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 msgr2=0x55c3ff7c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2e3fff700 1 -- 172.21.15.77:0/393035653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 msgr2=0x55c3ff7c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:33.919 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2e3fff700 1 -- 172.21.15.77:0/393035653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3ff7c5f40 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2e2ffd700 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2e3fff700 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c5950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0df7c0 tx=0x7fb2dc009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc522ede3dc0da4d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2c2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/393035653 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0ad790 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/393035653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d8005450 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/393035653 <== mon.2 v2:172.21.15.92:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7fb2dc0a8850 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/393035653 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc0b21b0 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2f4abeb80 10 client.25154 fetch_fsmap learned FSMap version 35 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/393035653 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c3ff82cef0 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/393035653 <== mon.2 v2:172.21.15.92:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2d8005450 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/393035653 <== mon.2 v2:172.21.15.92:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb2dc0a80b0 con 0x55c3ff7ac7b0 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2f4abeb80 10 client.25154 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2f4abeb80 10 client.25154 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:43:33.920 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2f4abeb80 20 client.25154 populate_metadata read hostname 'smithi077' 2024-05-27T08:43:33.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.915+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c3ff82d620 0x55c3ff82fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:33.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.916+0000 7fb2f4abeb80 4 client.25154 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-05-27T08:43:33.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.916+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/393035653 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55c3ff701630 con 0x55c3ff82d620 2024-05-27T08:43:33.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.916+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c3ff82d620 0x55c3ff82fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:33.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.916+0000 7fb2c2ffd700 10 client.25154 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:33.921 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:33.916+0000 7fb2e37fe700 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c3ff82d620 0x55c3ff82fa40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd3c3839c448769 server_cookie=9a392dba56f01126 in_seq=0 out_seq=0 2024-05-27T08:43:33.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:33 smithi092 ceph-mon[36049]: pgmap v161: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-05-27T08:43:34.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.004+0000 7fb2c2ffd700 1 -- 172.21.15.77:0/393035653 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c3ff701630 con 0x55c3ff82d620 2024-05-27T08:43:34.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.004+0000 7fb2c2ffd700 10 client.25154 handle_command_reply: tid=0 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.004+0000 7fb2f4abeb80 1 client.25154 shutdown 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.004+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 msgr2=0x55c3ff7c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.004+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c5950 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0df7c0 tx=0x7fb2dc009060 comp rx=0 tx=0).stop 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.004+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/393035653 shutdown_connections 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.004+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c3ff82d620 0x55c3ff82fa40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c3ff7ac7b0 0x55c3ff7c5950 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ff7a7c80 0x55c3ff7c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2f4abeb80 1 --2- 172.21.15.77:0/393035653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c3ff7a8640 0x55c3ff7c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/393035653 >> 172.21.15.77:0/393035653 conn(0x55c3ff6d33d0 msgr2=0x55c3ff7adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2c2ffd700 0 client.25154 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/393035653 shutdown_connections 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2f4abeb80 1 -- 172.21.15.77:0/393035653 wait complete. 2024-05-27T08:43:34.006 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2f4abeb80 20 client.25154 trim_cache size 0 max 16384 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 18, 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "cfcdb533": { 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 18, 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:43:34.007 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 144.5, 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:34.008 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:34.009 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:34.009 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:43:34.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2ea228700 1 -- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb2cc0b2b20 msgr2=0x7fb2cc0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2ea228700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb2cc0b2b20 0x7fb2cc0b4fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4108920 tx=0x7fb2d400b3c0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2ea228700 1 -- 172.21.15.77:0/71505870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e41073d0 msgr2=0x7fb2e4199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.005+0000 7fb2ea228700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e41073d0 0x7fb2e4199aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00ef30 tx=0x7fb2dc00c3a0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.006+0000 7fb2ea228700 1 -- 172.21.15.77:0/71505870 shutdown_connections 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.006+0000 7fb2ea228700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb2cc0b2b20 0x7fb2cc0b4fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.006+0000 7fb2ea228700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2e41073d0 0x7fb2e4199aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.006+0000 7fb2ea228700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb2e4102470 0x7fb2e419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.006+0000 7fb2ea228700 1 --2- 172.21.15.77:0/71505870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb2e4102e30 0x7fb2e419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.006+0000 7fb2ea228700 1 -- 172.21.15.77:0/71505870 >> 172.21.15.77:0/71505870 conn(0x7fb2e4079480 msgr2=0x7fb2e406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.006+0000 7fb2ea228700 1 -- 172.21.15.77:0/71505870 shutdown_connections 2024-05-27T08:43:34.010 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:34.006+0000 7fb2ea228700 1 -- 172.21.15.77:0/71505870 wait complete. 2024-05-27T08:43:34.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:33 smithi179 ceph-mon[36481]: pgmap v161: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 143 op/s 2024-05-27T08:43:34.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.96 MB/sec execute 88 sec latency 650.094 ms 2024-05-27T08:43:34.763 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cfcdb533', quiesce took 6.2 seconds. Will release after: 18.3 2024-05-27T08:43:35.600 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.93 MB/sec execute 89 sec latency 1650.125 ms 2024-05-27T08:43:35.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:35 smithi077 ceph-mon[32007]: pgmap v162: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-27T08:43:35.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:35 smithi092 ceph-mon[36049]: pgmap v162: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-27T08:43:36.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:35 smithi179 ceph-mon[36481]: pgmap v162: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-27T08:43:36.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.90 MB/sec execute 90 sec latency 2650.219 ms 2024-05-27T08:43:37.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.87 MB/sec execute 91 sec latency 3650.267 ms 2024-05-27T08:43:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:37 smithi092 ceph-mon[36049]: pgmap v163: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-27T08:43:38.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:37 smithi179 ceph-mon[36481]: pgmap v163: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-27T08:43:38.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:37 smithi077 ceph-mon[32007]: pgmap v163: 97 pgs: 97 active+clean; 89 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-05-27T08:43:38.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.83 MB/sec execute 92 sec latency 4650.312 ms 2024-05-27T08:43:39.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.80 MB/sec execute 93 sec latency 5650.370 ms 2024-05-27T08:43:39.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:39 smithi179 ceph-mon[36481]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-05-27T08:43:39.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:39 smithi092 ceph-mon[36049]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-05-27T08:43:40.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:39 smithi077 ceph-mon[32007]: pgmap v164: 97 pgs: 97 active+clean; 88 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-05-27T08:43:40.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.77 MB/sec execute 94 sec latency 6650.425 ms 2024-05-27T08:43:41.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.74 MB/sec execute 95 sec latency 7650.473 ms 2024-05-27T08:43:41.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:41 smithi092 ceph-mon[36049]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-05-27T08:43:42.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:41 smithi179 ceph-mon[36481]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-05-27T08:43:42.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:41 smithi077 ceph-mon[32007]: pgmap v165: 97 pgs: 97 active+clean; 88 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 160 op/s 2024-05-27T08:43:42.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.72 MB/sec execute 96 sec latency 8650.561 ms 2024-05-27T08:43:43.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.69 MB/sec execute 97 sec latency 9650.629 ms 2024-05-27T08:43:43.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:43 smithi092 ceph-mon[36049]: pgmap v166: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-05-27T08:43:44.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:43 smithi179 ceph-mon[36481]: pgmap v166: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-05-27T08:43:44.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:43 smithi077 ceph-mon[32007]: pgmap v166: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-05-27T08:43:44.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.66 MB/sec execute 98 sec latency 10650.690 ms 2024-05-27T08:43:45.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.63 MB/sec execute 99 sec latency 11650.731 ms 2024-05-27T08:43:45.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:45 smithi092 ceph-mon[36049]: pgmap v167: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 308 KiB/s wr, 66 op/s 2024-05-27T08:43:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:45 smithi179 ceph-mon[36481]: pgmap v167: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 308 KiB/s wr, 66 op/s 2024-05-27T08:43:46.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:45 smithi077 ceph-mon[32007]: pgmap v167: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 308 KiB/s wr, 66 op/s 2024-05-27T08:43:46.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.61 MB/sec execute 100 sec latency 12650.857 ms 2024-05-27T08:43:47.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.58 MB/sec execute 101 sec latency 13650.968 ms 2024-05-27T08:43:47.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:47 smithi092 ceph-mon[36049]: pgmap v168: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 308 KiB/s wr, 66 op/s 2024-05-27T08:43:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:47 smithi179 ceph-mon[36481]: pgmap v168: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 308 KiB/s wr, 66 op/s 2024-05-27T08:43:48.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:47 smithi077 ceph-mon[32007]: pgmap v168: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 308 KiB/s wr, 66 op/s 2024-05-27T08:43:48.601 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.56 MB/sec execute 102 sec latency 14651.082 ms 2024-05-27T08:43:49.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.53 MB/sec execute 103 sec latency 15651.198 ms 2024-05-27T08:43:49.931 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:49 smithi077 ceph-mon[32007]: pgmap v169: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 316 KiB/s wr, 72 op/s 2024-05-27T08:43:49.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:49 smithi092 ceph-mon[36049]: pgmap v169: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 316 KiB/s wr, 72 op/s 2024-05-27T08:43:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:49 smithi179 ceph-mon[36481]: pgmap v169: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 316 KiB/s wr, 72 op/s 2024-05-27T08:43:50.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.51 MB/sec execute 104 sec latency 16651.237 ms 2024-05-27T08:43:51.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.48 MB/sec execute 105 sec latency 17651.322 ms 2024-05-27T08:43:51.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:51 smithi092 ceph-mon[36049]: pgmap v170: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-05-27T08:43:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:51 smithi179 ceph-mon[36481]: pgmap v170: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-05-27T08:43:52.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:51 smithi077 ceph-mon[32007]: pgmap v170: 97 pgs: 97 active+clean; 88 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-05-27T08:43:52.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.46 MB/sec execute 106 sec latency 18651.374 ms 2024-05-27T08:43:53.064 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:43:53.323 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:43:53.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.44 MB/sec execute 107 sec latency 19651.422 ms 2024-05-27T08:43:53.732 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:53 smithi077 ceph-mon[32007]: pgmap v171: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 985 KiB/s wr, 38 op/s 2024-05-27T08:43:53.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:53 smithi092 ceph-mon[36049]: pgmap v171: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 985 KiB/s wr, 38 op/s 2024-05-27T08:43:54.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:53 smithi179 ceph-mon[36481]: pgmap v171: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 985 KiB/s wr, 38 op/s 2024-05-27T08:43:54.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.41 MB/sec execute 108 sec latency 20651.498 ms 2024-05-27T08:43:55.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 -- 172.21.15.77:0/1660884738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 msgr2=0x7f38ac106820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:55.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 --2- 172.21.15.77:0/1660884738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 0x7f38ac106820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f38a000aad0 tx=0x7f38a000ade0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 -- 172.21.15.77:0/1660884738 shutdown_connections 2024-05-27T08:43:55.165 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 --2- 172.21.15.77:0/1660884738 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38ac106f00 0x7f38ac10ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 --2- 172.21.15.77:0/1660884738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38ac101e20 0x7f38ac102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 --2- 172.21.15.77:0/1660884738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 0x7f38ac106820 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 -- 172.21.15.77:0/1660884738 >> 172.21.15.77:0/1660884738 conn(0x7f38ac078a40 msgr2=0x7f38ac0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 -- 172.21.15.77:0/1660884738 shutdown_connections 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.162+0000 7f38b251f700 1 -- 172.21.15.77:0/1660884738 wait complete. 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38b251f700 1 Processor -- start 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38b251f700 1 -- start start 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38b251f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38ac101e20 0x7f38ac19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38b251f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 0x7f38ac19dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:55.166 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38b251f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38ac106f00 0x7f38ac197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38b251f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1115e0 con 0x7f38ac1027d0 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38b251f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac111460 con 0x7f38ac106f00 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38b251f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac111760 con 0x7f38ac101e20 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38ab7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 0x7f38ac19dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38ab7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 0x7f38ac19dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42038/0 (socket says 172.21.15.77:42038) 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38ab7fe700 1 -- 172.21.15.77:0/62814174 learned_addr learned my addr 172.21.15.77:0/62814174 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.163+0000 7f38abfff700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38ac101e20 0x7f38ac19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38b0abc700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38ac106f00 0x7f38ac197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38ab7fe700 1 -- 172.21.15.77:0/62814174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38ac101e20 msgr2=0x7f38ac19d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38ab7fe700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38ac101e20 0x7f38ac19d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38ab7fe700 1 -- 172.21.15.77:0/62814174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38ac106f00 msgr2=0x7f38ac197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38ab7fe700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38ac106f00 0x7f38ac197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.167 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38ab7fe700 1 -- 172.21.15.77:0/62814174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac1980a0 con 0x7f38ac1027d0 2024-05-27T08:43:55.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38ab7fe700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 0x7f38ac19dcc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f38a00047e0 tx=0x7f38a000cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:55.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38a97fa700 1 -- 172.21.15.77:0/62814174 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0024020 con 0x7f38ac1027d0 2024-05-27T08:43:55.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38a97fa700 1 -- 172.21.15.77:0/62814174 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a0004280 con 0x7f38ac1027d0 2024-05-27T08:43:55.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38a97fa700 1 -- 172.21.15.77:0/62814174 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a001b040 con 0x7f38ac1027d0 2024-05-27T08:43:55.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac198330 con 0x7f38ac1027d0 2024-05-27T08:43:55.168 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.164+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac1a4650 con 0x7f38ac1027d0 2024-05-27T08:43:55.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.165+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac06a230 con 0x7f38ac1027d0 2024-05-27T08:43:55.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.167+0000 7f38a97fa700 1 -- 172.21.15.77:0/62814174 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f38a0027020 con 0x7f38ac1027d0 2024-05-27T08:43:55.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.167+0000 7f38a97fa700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f38940b29f0 0x7f38940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:55.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.168+0000 7f38abfff700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f38940b29f0 0x7f38940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:55.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.168+0000 7f38a97fa700 1 -- 172.21.15.77:0/62814174 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f38a00e5440 con 0x7f38ac1027d0 2024-05-27T08:43:55.170 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.168+0000 7f38abfff700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f38940b29f0 0x7f38940b4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f38ac198a80 tx=0x7f389c009190 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:55.171 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.171+0000 7f38a97fa700 1 -- 172.21.15.77:0/62814174 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f38a002e020 con 0x7f38ac1027d0 2024-05-27T08:43:55.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.424+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38ac02e560 con 0x7f38ac1027d0 2024-05-27T08:43:55.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.425+0000 7f38a97fa700 1 -- 172.21.15.77:0/62814174 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11927 (secure 0 0 0) 0x7f38a00acec0 con 0x7f38ac1027d0 2024-05-27T08:43:55.425 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:43:55.425 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:42:58.669173+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24418,24434,24410,24412,24433]},"id":1}]} 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f38940b29f0 msgr2=0x7f38940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f38940b29f0 0x7f38940b4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f38ac198a80 tx=0x7f389c009190 comp rx=0 tx=0).stop 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 msgr2=0x7f38ac19dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 0x7f38ac19dcc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f38a00047e0 tx=0x7f38a000cfb0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 shutdown_connections 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f38940b29f0 0x7f38940b4eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38ac106f00 0x7f38ac197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38ac101e20 0x7f38ac19d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 --2- 172.21.15.77:0/62814174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f38ac1027d0 0x7f38ac19dcc0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:55.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 >> 172.21.15.77:0/62814174 conn(0x7f38ac078a40 msgr2=0x7f38ac0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:55.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 shutdown_connections 2024-05-27T08:43:55.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:55.427+0000 7f38b251f700 1 -- 172.21.15.77:0/62814174 wait complete. 2024-05-27T08:43:55.429 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 35 2024-05-27T08:43:55.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.39 MB/sec execute 109 sec latency 21651.586 ms 2024-05-27T08:43:55.823 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:55 smithi077 ceph-mon[32007]: pgmap v172: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 966 KiB/s wr, 26 op/s 2024-05-27T08:43:55.824 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:55 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/62814174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:55.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:55 smithi092 ceph-mon[36049]: pgmap v172: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 966 KiB/s wr, 26 op/s 2024-05-27T08:43:55.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:55 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/62814174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:56.090 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id cfcdb533 --release --await' 2024-05-27T08:43:56.090 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db --set-id cfcdb533 --release --await 2024-05-27T08:43:56.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:55 smithi179 ceph-mon[36481]: pgmap v172: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 966 KiB/s wr, 26 op/s 2024-05-27T08:43:56.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:55 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/62814174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:43:56.336 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:43:56.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.37 MB/sec execute 110 sec latency 22651.621 ms 2024-05-27T08:43:57.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 137795 2.35 MB/sec execute 111 sec latency 23651.686 ms 2024-05-27T08:43:57.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 -- 172.21.15.77:0/3009842951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 msgr2=0x7f5ca010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/3009842951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 0x7f5ca010f2e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800b0a0 tx=0x7f5c9800b3b0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 -- 172.21.15.77:0/3009842951 shutdown_connections 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/3009842951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 0x7f5ca010f2e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/3009842951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca0108510 0x7f5ca01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/3009842951 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca0102430 0x7f5ca01028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 -- 172.21.15.77:0/3009842951 >> 172.21.15.77:0/3009842951 conn(0x7f5ca00793b0 msgr2=0x7f5ca00797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 -- 172.21.15.77:0/3009842951 shutdown_connections 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.823+0000 7f5ca4e79700 1 -- 172.21.15.77:0/3009842951 wait complete. 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.824+0000 7f5ca4e79700 1 Processor -- start 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.824+0000 7f5ca4e79700 1 -- start start 2024-05-27T08:43:57.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.824+0000 7f5ca4e79700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca0102430 0x7f5ca019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.824+0000 7f5ca4e79700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 0x7f5ca019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.824+0000 7f5ca4e79700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca0108510 0x7f5ca01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.824+0000 7f5ca4e79700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01125c0 con 0x7f5ca0102df0 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.824+0000 7f5ca4e79700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0112440 con 0x7f5ca0102430 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.824+0000 7f5ca4e79700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca0112740 con 0x7f5ca0108510 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 0x7f5ca019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 0x7f5ca019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42050/0 (socket says 172.21.15.77:42050) 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/982618413 learned_addr learned my addr 172.21.15.77:0/982618413 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca0102430 0x7f5ca019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9ed9d700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca0108510 0x7f5ca01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/982618413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca0108510 msgr2=0x7f5ca01998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca0108510 0x7f5ca01998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/982618413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca0102430 msgr2=0x7f5ca019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca0102430 0x7f5ca019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/982618413 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca019a000 con 0x7f5ca0102df0 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 0x7f5ca019fc20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400c9b0 tx=0x7f5c9400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c8f7fe700 1 -- 172.21.15.77:0/982618413 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94007b00 con 0x7f5ca0102df0 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c8f7fe700 1 -- 172.21.15.77:0/982618413 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c94007ca0 con 0x7f5ca0102df0 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5ca4e79700 1 -- 172.21.15.77:0/982618413 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca019a2f0 con 0x7f5ca0102df0 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5c8f7fe700 1 -- 172.21.15.77:0/982618413 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94018660 con 0x7f5ca0102df0 2024-05-27T08:43:57.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.825+0000 7f5ca4e79700 1 -- 172.21.15.77:0/982618413 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca01a65b0 con 0x7f5ca0102df0 2024-05-27T08:43:57.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.828+0000 7f5c8f7fe700 1 -- 172.21.15.77:0/982618413 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5c9400f450 con 0x7f5ca0102df0 2024-05-27T08:43:57.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.828+0000 7f5c8f7fe700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5c880b29f0 0x7f5c880b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.828+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5c880b29f0 0x7f5c880b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.828+0000 7f5c8f7fe700 1 -- 172.21.15.77:0/982618413 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5c940e25f0 con 0x7f5ca0102df0 2024-05-27T08:43:57.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.829+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5c880b29f0 0x7f5c880b4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ca019a9e0 tx=0x7f5c9000a380 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:57.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.829+0000 7f5caf70fb80 1 Processor -- start 2024-05-27T08:43:57.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.829+0000 7f5caf70fb80 1 -- start start 2024-05-27T08:43:57.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.829+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da37720 0x561c8da37b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.829+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da33070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.829+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da335b0 0x561c8da3f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c8d990e70 con 0x561c8da37720 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561c8d980a10 con 0x561c8da335b0 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561c8d984a50 con 0x561c8da32bf0 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da37720 0x561c8da37b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da37720 0x561c8da37b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42064/0 (socket says 172.21.15.77:42064) 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 -- 172.21.15.77:0/1701397559 learned_addr learned my addr 172.21.15.77:0/1701397559 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da33070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 -- 172.21.15.77:0/1701397559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 msgr2=0x561c8da33070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:57.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da33070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9ed9d700 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da335b0 0x561c8da3f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 -- 172.21.15.77:0/1701397559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da335b0 msgr2=0x561c8da3f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da335b0 0x561c8da3f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 -- 172.21.15.77:0/1701397559 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c8d92a0b0 con 0x561c8da37720 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9ed9d700 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da335b0 0x561c8da3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da37720 0x561c8da37b00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5c90004530 tx=0x7f5c90012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f42847ab30f2bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c8cff9700 1 -- 172.21.15.77:0/1701397559 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90016070 con 0x561c8da37720 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c8cff9700 1 -- 172.21.15.77:0/1701397559 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c90014070 con 0x561c8da37720 2024-05-27T08:43:57.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.830+0000 7f5c8cff9700 1 -- 172.21.15.77:0/1701397559 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90010e00 con 0x561c8da37720 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1701397559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da37720 msgr2=0x561c8da37b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da37720 0x561c8da37b00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5c90004530 tx=0x7f5c90012000 comp rx=0 tx=0).stop 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1701397559 shutdown_connections 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da335b0 0x561c8da3f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da37720 0x561c8da37b00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/1701397559 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da33070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1701397559 >> 172.21.15.77:0/1701397559 conn(0x561c8d95e340 msgr2=0x561c8da2d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1701397559 shutdown_connections 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.831+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1701397559 wait complete. 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 1 -- start start 2024-05-27T08:43:57.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 1 Processor -- start 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da3f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da3c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 0x561c8da3caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c8d8870a0 con 0x561c8da335b0 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x561c8d8e8ea0 con 0x561c8da37720 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561c8da3fc80 con 0x561c8da32bf0 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5caf70fb80 10 client.0 _ll_register_callbacks cb 0x561c8d91ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da3c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da3c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42068/0 (socket says 172.21.15.77:42068) 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/893263492 learned_addr learned my addr 172.21.15.77:0/893263492 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:57.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9ed9d700 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 0x561c8da3caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da3f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/893263492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 msgr2=0x561c8da3f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da3f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/893263492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 msgr2=0x561c8da3caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 0x561c8da3caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/893263492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c8d8d09d0 con 0x561c8da335b0 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da3f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da3c5b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5c940e0350 tx=0x7f5c940a96b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d160d4a114667b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.832+0000 7f5c85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5c85ffb700 1 -- 172.21.15.77:0/893263492 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c940c3060 con 0x561c8da335b0 2024-05-27T08:43:57.841 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5c85ffb700 1 -- 172.21.15.77:0/893263492 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5c940a5ed0 con 0x561c8da335b0 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 1 -- 172.21.15.77:0/893263492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c90014070 con 0x561c8da335b0 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5c85ffb700 1 -- 172.21.15.77:0/893263492 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5c940a9050 con 0x561c8da335b0 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 10 client.15669 fetch_fsmap learned FSMap version 35 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 1 -- 172.21.15.77:0/893263492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561c8d93a860 con 0x561c8da335b0 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5c85ffb700 1 -- 172.21.15.77:0/893263492 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c90014070 con 0x561c8da335b0 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5c85ffb700 1 -- 172.21.15.77:0/893263492 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f5c9406f1a0 con 0x561c8da335b0 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 10 client.15669 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 10 client.15669 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 20 client.15669 populate_metadata read hostname 'smithi077' 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x561c8da4c7f0 0x561c8da4ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.842 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 4 client.15669 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5caf70fb80 1 -- 172.21.15.77:0/893263492 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c8d992dc0 con 0x561c8da4c7f0 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.833+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x561c8da4c7f0 0x561c8da4ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.834+0000 7f5c85ffb700 10 client.15669 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.834+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x561c8da4c7f0 0x561c8da4ebe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e5a978a21bc0e53 server_cookie=4c8d9e5c98fa715c in_seq=0 out_seq=0 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5c85ffb700 1 -- 172.21.15.77:0/893263492 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561c8d992dc0 con 0x561c8da4c7f0 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5c85ffb700 10 client.15669 handle_command_reply: tid=0 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 client.15669 shutdown 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 -- 172.21.15.77:0/893263492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 msgr2=0x561c8da3c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da3c5b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5c940e0350 tx=0x7f5c940a96b0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 -- 172.21.15.77:0/893263492 shutdown_connections 2024-05-27T08:43:57.843 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x561c8da4c7f0 0x561c8da4ebe0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 0x561c8da3caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5c85ffb700 0 client.15669 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:57.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da32bf0 0x561c8da3f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/893263492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da3c5b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 -- 172.21.15.77:0/893263492 >> 172.21.15.77:0/893263492 conn(0x561c8d95e340 msgr2=0x561c8da31be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:57.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 -- 172.21.15.77:0/893263492 shutdown_connections 2024-05-27T08:43:57.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 1 -- 172.21.15.77:0/893263492 wait complete. 2024-05-27T08:43:57.844 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.835+0000 7f5caf70fb80 20 client.15669 trim_cache size 0 max 16384 2024-05-27T08:43:57.856 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5caf70fb80 1 Processor -- start 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5caf70fb80 1 -- start start 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da32bf0 0x561c8da27e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da335b0 0x561c8da36dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 0x561c8da54410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01adce0 con 0x561c8da32bf0 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01af820 con 0x561c8da37720 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01ae3b0 con 0x561c8da335b0 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da32bf0 0x561c8da27e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9e59c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da32bf0 0x561c8da27e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42072/0 (socket says 172.21.15.77:42072) 2024-05-27T08:43:57.857 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9e59c700 1 -- 172.21.15.77:0/1835779191 learned_addr learned my addr 172.21.15.77:0/1835779191 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/1835779191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da335b0 0x561c8da36dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9e59c700 1 -- 172.21.15.77:0/1835779191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da335b0 msgr2=0x561c8da36dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/1835779191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da335b0 0x561c8da36dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9e59c700 1 -- 172.21.15.77:0/1835779191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 msgr2=0x561c8da54410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/1835779191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 0x561c8da54410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.855+0000 7f5c9e59c700 1 -- 172.21.15.77:0/1835779191 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c8da33c30 con 0x561c8da32bf0 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/1835779191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da32bf0 0x561c8da27e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000ff80 tx=0x7f5c90005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63286d2a598dba18 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5c87fff700 1 -- 172.21.15.77:0/1835779191 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90035070 con 0x561c8da32bf0 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5c87fff700 1 -- 172.21.15.77:0/1835779191 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c90014050 con 0x561c8da32bf0 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5c87fff700 1 -- 172.21.15.77:0/1835779191 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90016070 con 0x561c8da32bf0 2024-05-27T08:43:57.858 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1835779191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da32bf0 msgr2=0x561c8da27e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/1835779191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da32bf0 0x561c8da27e70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000ff80 tx=0x7f5c90005e50 comp rx=0 tx=0).stop 2024-05-27T08:43:57.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1835779191 shutdown_connections 2024-05-27T08:43:57.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/1835779191 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da37720 0x561c8da54410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/1835779191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da32bf0 0x561c8da27e70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/1835779191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da335b0 0x561c8da36dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1835779191 >> 172.21.15.77:0/1835779191 conn(0x561c8d95e340 msgr2=0x561c8da366c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:57.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1835779191 shutdown_connections 2024-05-27T08:43:57.859 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.856+0000 7f5caf70fb80 1 -- 172.21.15.77:0/1835779191 wait complete. 2024-05-27T08:43:57.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 1 -- start start 2024-05-27T08:43:57.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 1 Processor -- start 2024-05-27T08:43:57.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da32bf0 0x561c8da414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da41a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da37720 0x561c8daba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01af150 con 0x561c8da335b0 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01aea80 con 0x561c8da32bf0 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca01abc80 con 0x561c8da37720 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5caf70fb80 10 client.0 _ll_register_callbacks cb 0x561c8d91ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da41a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9dd9b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da41a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:42078/0 (socket says 172.21.15.77:42078) 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/3303773079 learned_addr learned my addr 172.21.15.77:0/3303773079 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9ed9d700 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da37720 0x561c8daba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da32bf0 0x561c8da414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/3303773079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da37720 msgr2=0x561c8daba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:57.863 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da37720 0x561c8daba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/3303773079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da32bf0 msgr2=0x561c8da414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da32bf0 0x561c8da414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9dd9b700 1 -- 172.21.15.77:0/3303773079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c8dabade0 con 0x561c8da335b0 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.857+0000 7f5c9ed9d700 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da37720 0x561c8daba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5c9dd9b700 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da41a20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5c940a6b50 tx=0x7f5c940cc370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce47ba6cd5250e97 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5c85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5c85ffb700 1 -- 172.21.15.77:0/3303773079 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c940c3020 con 0x561c8da335b0 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5c85ffb700 1 -- 172.21.15.77:0/3303773079 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8498+0+0 (secure 0 0 0) 0x7f5c940c5020 con 0x561c8da335b0 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 1 -- 172.21.15.77:0/3303773079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c90014050 con 0x561c8da335b0 2024-05-27T08:43:57.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5c85ffb700 1 -- 172.21.15.77:0/3303773079 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5c940b2030 con 0x561c8da335b0 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 10 client.15675 fetch_fsmap learned FSMap version 35 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 1 -- 172.21.15.77:0/3303773079 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561c8dabc0e0 con 0x561c8da335b0 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5c85ffb700 1 -- 172.21.15.77:0/3303773079 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c90014050 con 0x561c8da335b0 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5c85ffb700 1 -- 172.21.15.77:0/3303773079 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f5c940e20d0 con 0x561c8da335b0 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 10 client.15675 fetch_fsmap finished waiting for FSMap version 35 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 10 client.15675 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 20 client.15675 populate_metadata read hostname 'smithi077' 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x561c8dabc7c0 0x561c8dabebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 4 client.15675 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cfcdb533", "release": true, "await": true}] 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5caf70fb80 1 -- 172.21.15.77:0/3303773079 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cfcdb533", "release": true, "await": true}) -- 0x561c8d98c5a0 con 0x561c8dabc7c0 2024-05-27T08:43:57.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.858+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x561c8dabc7c0 0x561c8dabebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:43:57.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.859+0000 7f5c9e59c700 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x561c8dabc7c0 0x561c8dabebe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc5a8c4747f7f190 server_cookie=37c8811d3a08278c in_seq=0 out_seq=0 2024-05-27T08:43:57.866 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.859+0000 7f5c85ffb700 10 client.15675 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:57.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5c85ffb700 1 -- 172.21.15.77:0/3303773079 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561c8d98c5a0 con 0x561c8dabc7c0 2024-05-27T08:43:57.870 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5c85ffb700 10 client.15675 handle_command_reply: tid=0 2024-05-27T08:43:57.871 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:57 smithi077 ceph-mon[32007]: pgmap v173: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 966 KiB/s wr, 26 op/s 2024-05-27T08:43:57.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 client.15675 shutdown 2024-05-27T08:43:57.871 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 -- 172.21.15.77:0/3303773079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 msgr2=0x561c8da41a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 35, 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 20, 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout: "cfcdb533": { 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 20, 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:43:57.872 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 144.5, 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:43:57.873 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:43:57.874 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:43:57.874 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:43:57.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:57.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:57.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:57.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:57.874 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:43:57.875 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:43:57.875 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da41a20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5c940a6b50 tx=0x7f5c940cc370 comp rx=0 tx=0).stop 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 -- 172.21.15.77:0/3303773079 shutdown_connections 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x561c8dabc7c0 0x561c8dabebe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c8da37720 0x561c8daba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x561c8da32bf0 0x561c8da414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 --2- 172.21.15.77:0/3303773079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c8da335b0 0x561c8da41a20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 -- 172.21.15.77:0/3303773079 >> 172.21.15.77:0/3303773079 conn(0x561c8d95e340 msgr2=0x561c8da3dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5c85ffb700 0 client.15675 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 -- 172.21.15.77:0/3303773079 shutdown_connections 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.870+0000 7f5caf70fb80 1 -- 172.21.15.77:0/3303773079 wait complete. 2024-05-27T08:43:57.876 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5caf70fb80 20 client.15675 trim_cache size 0 max 16384 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 -- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5c880b29f0 msgr2=0x7f5c880b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5c880b29f0 0x7f5c880b4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ca019a9e0 tx=0x7f5c9000a380 comp rx=0 tx=0).stop 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 -- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 msgr2=0x7f5ca019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 0x7f5ca019fc20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400c9b0 tx=0x7f5c9400ccc0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 -- 172.21.15.77:0/982618413 shutdown_connections 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5c880b29f0 0x7f5c880b4eb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ca0108510 0x7f5ca01998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5ca0102430 0x7f5ca019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 --2- 172.21.15.77:0/982618413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ca0102df0 0x7f5ca019fc20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 -- 172.21.15.77:0/982618413 >> 172.21.15.77:0/982618413 conn(0x7f5ca00793b0 msgr2=0x7f5ca006bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:43:57.877 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 -- 172.21.15.77:0/982618413 shutdown_connections 2024-05-27T08:43:57.878 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:43:57.871+0000 7f5ca4e79700 1 -- 172.21.15.77:0/982618413 wait complete. 2024-05-27T08:43:57.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:57 smithi092 ceph-mon[36049]: pgmap v173: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 966 KiB/s wr, 26 op/s 2024-05-27T08:43:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:57 smithi179 ceph-mon[36481]: pgmap v173: 97 pgs: 97 active+clean; 98 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 966 KiB/s wr, 26 op/s 2024-05-27T08:43:58.549 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cfcdb533', total seconds elapsed: 29.9 2024-05-27T08:43:58.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 138824 2.39 MB/sec execute 112 sec latency 23921.319 ms 2024-05-27T08:43:59.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 141246 2.47 MB/sec execute 113 sec latency 10.902 ms 2024-05-27T08:43:59.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:43:59 smithi092 ceph-mon[36049]: pgmap v174: 97 pgs: 97 active+clean; 89 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 49 op/s 2024-05-27T08:44:00.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:43:59 smithi179 ceph-mon[36481]: pgmap v174: 97 pgs: 97 active+clean; 89 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 49 op/s 2024-05-27T08:44:00.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:43:59 smithi077 ceph-mon[32007]: pgmap v174: 97 pgs: 97 active+clean; 89 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 49 op/s 2024-05-27T08:44:00.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 142197 2.51 MB/sec execute 114 sec latency 460.270 ms 2024-05-27T08:44:01.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 142197 2.49 MB/sec execute 115 sec latency 1460.334 ms 2024-05-27T08:44:01.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:01 smithi092 ceph-mon[36049]: pgmap v175: 97 pgs: 97 active+clean; 89 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1012 KiB/s wr, 42 op/s 2024-05-27T08:44:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:01 smithi179 ceph-mon[36481]: pgmap v175: 97 pgs: 97 active+clean; 89 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1012 KiB/s wr, 42 op/s 2024-05-27T08:44:02.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:01 smithi077 ceph-mon[32007]: pgmap v175: 97 pgs: 97 active+clean; 89 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 1012 KiB/s wr, 42 op/s 2024-05-27T08:44:02.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 142197 2.47 MB/sec execute 116 sec latency 2460.410 ms 2024-05-27T08:44:03.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 142306 2.45 MB/sec execute 117 sec latency 3299.658 ms 2024-05-27T08:44:03.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:03 smithi092 ceph-mon[36049]: pgmap v176: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-27T08:44:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:03 smithi179 ceph-mon[36481]: pgmap v176: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-27T08:44:04.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:03 smithi077 ceph-mon[32007]: pgmap v176: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-05-27T08:44:04.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 143510 2.47 MB/sec execute 118 sec latency 21.772 ms 2024-05-27T08:44:05.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 145754 2.56 MB/sec execute 119 sec latency 11.693 ms 2024-05-27T08:44:05.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:05 smithi092 ceph-mon[36049]: pgmap v177: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 358 KiB/s wr, 82 op/s 2024-05-27T08:44:06.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:05 smithi179 ceph-mon[36481]: pgmap v177: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 358 KiB/s wr, 82 op/s 2024-05-27T08:44:06.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:05 smithi077 ceph-mon[32007]: pgmap v177: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 358 KiB/s wr, 82 op/s 2024-05-27T08:44:06.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 147845 2.60 MB/sec execute 120 sec latency 12.401 ms 2024-05-27T08:44:07.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 149436 2.68 MB/sec execute 121 sec latency 11.626 ms 2024-05-27T08:44:07.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:07 smithi092 ceph-mon[36049]: pgmap v178: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 358 KiB/s wr, 82 op/s 2024-05-27T08:44:08.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:07 smithi179 ceph-mon[36481]: pgmap v178: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 358 KiB/s wr, 82 op/s 2024-05-27T08:44:08.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:07 smithi077 ceph-mon[32007]: pgmap v178: 97 pgs: 97 active+clean; 90 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 358 KiB/s wr, 82 op/s 2024-05-27T08:44:08.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 151761 2.75 MB/sec execute 122 sec latency 15.539 ms 2024-05-27T08:44:09.602 INFO:tasks.workunit.client.0.smithi077.stdout: 1 153217 2.80 MB/sec execute 123 sec latency 12.637 ms 2024-05-27T08:44:09.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:09 smithi092 ceph-mon[36049]: pgmap v179: 97 pgs: 97 active+clean; 95 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 978 KiB/s wr, 187 op/s 2024-05-27T08:44:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:09 smithi179 ceph-mon[36481]: pgmap v179: 97 pgs: 97 active+clean; 95 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 978 KiB/s wr, 187 op/s 2024-05-27T08:44:10.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:09 smithi077 ceph-mon[32007]: pgmap v179: 97 pgs: 97 active+clean; 95 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 978 KiB/s wr, 187 op/s 2024-05-27T08:44:10.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 155539 2.87 MB/sec execute 124 sec latency 79.305 ms 2024-05-27T08:44:11.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 156898 2.91 MB/sec execute 125 sec latency 15.826 ms 2024-05-27T08:44:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:11 smithi092 ceph-mon[36049]: pgmap v180: 97 pgs: 97 active+clean; 95 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 924 KiB/s wr, 165 op/s 2024-05-27T08:44:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:11 smithi179 ceph-mon[36481]: pgmap v180: 97 pgs: 97 active+clean; 95 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 924 KiB/s wr, 165 op/s 2024-05-27T08:44:12.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:11 smithi077 ceph-mon[32007]: pgmap v180: 97 pgs: 97 active+clean; 95 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 924 KiB/s wr, 165 op/s 2024-05-27T08:44:12.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 159449 3.00 MB/sec execute 126 sec latency 14.860 ms 2024-05-27T08:44:13.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 160781 3.02 MB/sec execute 127 sec latency 12.787 ms 2024-05-27T08:44:13.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:13 smithi092 ceph-mon[36049]: pgmap v181: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 269 op/s 2024-05-27T08:44:14.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:13 smithi179 ceph-mon[36481]: pgmap v181: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 269 op/s 2024-05-27T08:44:14.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:13 smithi077 ceph-mon[32007]: pgmap v181: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 269 op/s 2024-05-27T08:44:14.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 163395 3.13 MB/sec execute 128 sec latency 10.106 ms 2024-05-27T08:44:15.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 164679 3.15 MB/sec execute 129 sec latency 12.067 ms 2024-05-27T08:44:15.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:15 smithi092 ceph-mon[36049]: pgmap v182: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-05-27T08:44:16.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:15 smithi179 ceph-mon[36481]: pgmap v182: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-05-27T08:44:16.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:15 smithi077 ceph-mon[32007]: pgmap v182: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-05-27T08:44:16.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 167011 3.23 MB/sec execute 130 sec latency 10.386 ms 2024-05-27T08:44:17.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 167325 3.21 MB/sec execute 131 sec latency 616.125 ms 2024-05-27T08:44:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:17 smithi092 ceph-mon[36049]: pgmap v183: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-05-27T08:44:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:17 smithi179 ceph-mon[36481]: pgmap v183: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-05-27T08:44:18.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:17 smithi077 ceph-mon[32007]: pgmap v183: 97 pgs: 97 active+clean; 102 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 209 op/s 2024-05-27T08:44:18.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 167325 3.18 MB/sec execute 132 sec latency 1616.216 ms 2024-05-27T08:44:19.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 167325 3.16 MB/sec execute 133 sec latency 2616.276 ms 2024-05-27T08:44:19.946 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:19 smithi077 ceph-mon[32007]: pgmap v184: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-27T08:44:19.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:19 smithi092 ceph-mon[36049]: pgmap v184: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-27T08:44:20.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:19 smithi179 ceph-mon[36481]: pgmap v184: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 310 op/s 2024-05-27T08:44:20.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 167325 3.14 MB/sec execute 134 sec latency 3616.321 ms 2024-05-27T08:44:21.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 169162 3.17 MB/sec execute 135 sec latency 3716.656 ms 2024-05-27T08:44:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:21 smithi092 ceph-mon[36049]: pgmap v185: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-05-27T08:44:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:21 smithi179 ceph-mon[36481]: pgmap v185: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-05-27T08:44:22.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:21 smithi077 ceph-mon[32007]: pgmap v185: 97 pgs: 97 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 205 op/s 2024-05-27T08:44:22.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 170800 3.23 MB/sec execute 136 sec latency 9.969 ms 2024-05-27T08:44:23.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 172703 3.26 MB/sec execute 137 sec latency 17.736 ms 2024-05-27T08:44:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:23 smithi092 ceph-mon[36049]: pgmap v186: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-05-27T08:44:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:23 smithi179 ceph-mon[36481]: pgmap v186: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-05-27T08:44:24.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:23 smithi077 ceph-mon[32007]: pgmap v186: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-05-27T08:44:24.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 174299 3.32 MB/sec execute 138 sec latency 9.986 ms 2024-05-27T08:44:25.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 176123 3.35 MB/sec execute 139 sec latency 18.012 ms 2024-05-27T08:44:25.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:25 smithi092 ceph-mon[36049]: pgmap v187: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-27T08:44:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:25 smithi179 ceph-mon[36481]: pgmap v187: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-27T08:44:26.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:25 smithi077 ceph-mon[32007]: pgmap v187: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-27T08:44:26.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 177782 3.41 MB/sec execute 140 sec latency 12.073 ms 2024-05-27T08:44:27.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 179070 3.42 MB/sec execute 141 sec latency 22.755 ms 2024-05-27T08:44:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:27 smithi092 ceph-mon[36049]: pgmap v188: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-27T08:44:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:27 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:44:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:27 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:44:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:27 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:44:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:27 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:44:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:27 smithi179 ceph-mon[36481]: pgmap v188: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-27T08:44:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:27 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:44:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:27 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:44:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:27 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:44:28.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:27 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:44:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:27 smithi077 ceph-mon[32007]: pgmap v188: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-05-27T08:44:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:27 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:44:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:27 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:44:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:27 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:44:28.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:27 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:44:28.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 181222 3.50 MB/sec execute 142 sec latency 13.419 ms 2024-05-27T08:44:29.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 182604 3.51 MB/sec execute 143 sec latency 29.624 ms 2024-05-27T08:44:29.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:29 smithi092 ceph-mon[36049]: pgmap v189: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-05-27T08:44:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:29 smithi179 ceph-mon[36481]: pgmap v189: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-05-27T08:44:30.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:29 smithi077 ceph-mon[32007]: pgmap v189: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 246 op/s 2024-05-27T08:44:30.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 184766 3.58 MB/sec execute 144 sec latency 14.047 ms 2024-05-27T08:44:31.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 186232 3.59 MB/sec execute 145 sec latency 15.576 ms 2024-05-27T08:44:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:31 smithi092 ceph-mon[36049]: pgmap v190: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-05-27T08:44:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:31 smithi179 ceph-mon[36481]: pgmap v190: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-05-27T08:44:32.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:31 smithi077 ceph-mon[32007]: pgmap v190: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 145 op/s 2024-05-27T08:44:32.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 188323 3.66 MB/sec execute 146 sec latency 13.076 ms 2024-05-27T08:44:33.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 189615 3.67 MB/sec execute 147 sec latency 19.320 ms 2024-05-27T08:44:33.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:33 smithi092 ceph-mon[36049]: pgmap v191: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2024-05-27T08:44:34.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:33 smithi179 ceph-mon[36481]: pgmap v191: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2024-05-27T08:44:34.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:33 smithi077 ceph-mon[32007]: pgmap v191: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.9 MiB/s wr, 244 op/s 2024-05-27T08:44:34.603 INFO:tasks.workunit.client.0.smithi077.stdout: 1 191792 3.74 MB/sec execute 148 sec latency 19.113 ms 2024-05-27T08:44:35.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 193215 3.75 MB/sec execute 149 sec latency 18.596 ms 2024-05-27T08:44:35.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:35 smithi092 ceph-mon[36049]: pgmap v192: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-05-27T08:44:36.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:35 smithi179 ceph-mon[36481]: pgmap v192: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-05-27T08:44:36.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:35 smithi077 ceph-mon[32007]: pgmap v192: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-05-27T08:44:36.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 195415 3.82 MB/sec execute 150 sec latency 10.307 ms 2024-05-27T08:44:37.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 196717 3.83 MB/sec execute 151 sec latency 13.314 ms 2024-05-27T08:44:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:37 smithi092 ceph-mon[36049]: pgmap v193: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-05-27T08:44:38.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:37 smithi179 ceph-mon[36481]: pgmap v193: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-05-27T08:44:38.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:37 smithi077 ceph-mon[32007]: pgmap v193: 97 pgs: 97 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-05-27T08:44:38.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 198797 3.89 MB/sec execute 152 sec latency 13.412 ms 2024-05-27T08:44:39.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 200024 3.90 MB/sec execute 153 sec latency 48.681 ms 2024-05-27T08:44:39.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:39 smithi092 ceph-mon[36049]: pgmap v194: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-05-27T08:44:39.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:39 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:44:40.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:39 smithi179 ceph-mon[36481]: pgmap v194: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-05-27T08:44:40.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:39 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:44:40.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:39 smithi077 ceph-mon[32007]: pgmap v194: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-05-27T08:44:40.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:39 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:44:40.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 202291 3.96 MB/sec execute 154 sec latency 19.902 ms 2024-05-27T08:44:41.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 203443 3.96 MB/sec execute 155 sec latency 14.775 ms 2024-05-27T08:44:41.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:41 smithi092 ceph-mon[36049]: pgmap v195: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-05-27T08:44:42.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:41 smithi179 ceph-mon[36481]: pgmap v195: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-05-27T08:44:42.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:41 smithi077 ceph-mon[32007]: pgmap v195: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-05-27T08:44:42.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 205397 4.00 MB/sec execute 156 sec latency 14.939 ms 2024-05-27T08:44:43.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 206644 4.02 MB/sec execute 157 sec latency 16.067 ms 2024-05-27T08:44:43.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:43 smithi092 ceph-mon[36049]: pgmap v196: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-05-27T08:44:44.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:43 smithi179 ceph-mon[36481]: pgmap v196: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-05-27T08:44:44.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:43 smithi077 ceph-mon[32007]: pgmap v196: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 326 op/s 2024-05-27T08:44:44.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 208324 4.04 MB/sec execute 158 sec latency 38.714 ms 2024-05-27T08:44:44.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:44 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:44:45.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:44 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:44:45.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:44 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:44:45.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 209830 4.09 MB/sec execute 159 sec latency 22.665 ms 2024-05-27T08:44:45.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:45 smithi092 ceph-mon[36049]: pgmap v197: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-27T08:44:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:45 smithi179 ceph-mon[36481]: pgmap v197: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-27T08:44:46.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:45 smithi077 ceph-mon[32007]: pgmap v197: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-27T08:44:46.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 211746 4.11 MB/sec execute 160 sec latency 16.719 ms 2024-05-27T08:44:47.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 213291 4.15 MB/sec execute 161 sec latency 20.996 ms 2024-05-27T08:44:47.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:47 smithi092 ceph-mon[36049]: pgmap v198: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-27T08:44:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:47 smithi179 ceph-mon[36481]: pgmap v198: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-27T08:44:48.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:47 smithi077 ceph-mon[32007]: pgmap v198: 97 pgs: 97 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 227 op/s 2024-05-27T08:44:48.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 214694 4.16 MB/sec execute 162 sec latency 20.301 ms 2024-05-27T08:44:49.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 216707 4.22 MB/sec execute 163 sec latency 23.676 ms 2024-05-27T08:44:49.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:49 smithi092 ceph-mon[36049]: pgmap v199: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-27T08:44:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:49 smithi179 ceph-mon[36481]: pgmap v199: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-27T08:44:50.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:49 smithi077 ceph-mon[32007]: pgmap v199: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 351 op/s 2024-05-27T08:44:50.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 218117 4.23 MB/sec execute 164 sec latency 20.823 ms 2024-05-27T08:44:51.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 220279 4.28 MB/sec execute 165 sec latency 13.724 ms 2024-05-27T08:44:51.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:51 smithi092 ceph-mon[36049]: pgmap v200: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-05-27T08:44:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:51 smithi179 ceph-mon[36481]: pgmap v200: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-05-27T08:44:52.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:51 smithi077 ceph-mon[32007]: pgmap v200: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-05-27T08:44:52.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 221741 4.29 MB/sec execute 166 sec latency 16.536 ms 2024-05-27T08:44:53.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 223814 4.35 MB/sec execute 167 sec latency 14.692 ms 2024-05-27T08:44:53.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:53 smithi092 ceph-mon[36049]: pgmap v201: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 353 op/s 2024-05-27T08:44:54.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:53 smithi179 ceph-mon[36481]: pgmap v201: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 353 op/s 2024-05-27T08:44:54.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:53 smithi077 ceph-mon[32007]: pgmap v201: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 353 op/s 2024-05-27T08:44:54.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 225036 4.35 MB/sec execute 168 sec latency 24.777 ms 2024-05-27T08:44:55.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 227259 4.41 MB/sec execute 169 sec latency 21.353 ms 2024-05-27T08:44:55.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:55 smithi092 ceph-mon[36049]: pgmap v202: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-05-27T08:44:56.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:55 smithi179 ceph-mon[36481]: pgmap v202: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-05-27T08:44:56.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:55 smithi077 ceph-mon[32007]: pgmap v202: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-05-27T08:44:56.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 228512 4.41 MB/sec execute 170 sec latency 18.267 ms 2024-05-27T08:44:57.604 INFO:tasks.workunit.client.0.smithi077.stdout: 1 230759 4.46 MB/sec execute 171 sec latency 16.018 ms 2024-05-27T08:44:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:57 smithi179 ceph-mon[36481]: pgmap v203: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-05-27T08:44:58.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:57 smithi077 ceph-mon[32007]: pgmap v203: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-05-27T08:44:58.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:57 smithi092 ceph-mon[36049]: pgmap v203: 97 pgs: 97 active+clean; 148 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-05-27T08:44:58.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 232040 4.47 MB/sec execute 172 sec latency 17.767 ms 2024-05-27T08:44:59.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 233585 4.49 MB/sec execute 173 sec latency 15.240 ms 2024-05-27T08:45:00.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:44:59 smithi179 ceph-mon[36481]: pgmap v204: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 346 op/s 2024-05-27T08:45:00.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:44:59 smithi077 ceph-mon[32007]: pgmap v204: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 346 op/s 2024-05-27T08:45:00.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:44:59 smithi092 ceph-mon[36049]: pgmap v204: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 346 op/s 2024-05-27T08:45:00.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 234824 4.50 MB/sec execute 174 sec latency 31.519 ms 2024-05-27T08:45:01.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 236147 4.51 MB/sec execute 175 sec latency 23.762 ms 2024-05-27T08:45:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:01 smithi179 ceph-mon[36481]: pgmap v205: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-05-27T08:45:02.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:01 smithi077 ceph-mon[32007]: pgmap v205: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-05-27T08:45:02.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:01 smithi092 ceph-mon[36049]: pgmap v205: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-05-27T08:45:02.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 238060 4.56 MB/sec execute 176 sec latency 11.747 ms 2024-05-27T08:45:03.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 239323 4.56 MB/sec execute 177 sec latency 14.092 ms 2024-05-27T08:45:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:03 smithi179 ceph-mon[36481]: pgmap v206: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-05-27T08:45:04.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:03 smithi077 ceph-mon[32007]: pgmap v206: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-05-27T08:45:04.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:03 smithi092 ceph-mon[36049]: pgmap v206: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-05-27T08:45:04.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 241473 4.62 MB/sec execute 178 sec latency 15.864 ms 2024-05-27T08:45:05.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:04 smithi179 ceph-mon[36481]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T08:45:05.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:04 smithi077 ceph-mon[32007]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T08:45:05.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:04 smithi092 ceph-mon[36049]: osdmap e87: 12 total, 12 up, 12 in 2024-05-27T08:45:05.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 242762 4.62 MB/sec execute 179 sec latency 40.580 ms 2024-05-27T08:45:06.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:05 smithi179 ceph-mon[36481]: pgmap v208: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-05-27T08:45:06.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:05 smithi077 ceph-mon[32007]: pgmap v208: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-05-27T08:45:06.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:05 smithi092 ceph-mon[36049]: pgmap v208: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-05-27T08:45:06.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 244995 4.67 MB/sec execute 180 sec latency 18.151 ms 2024-05-27T08:45:07.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 246305 4.67 MB/sec execute 181 sec latency 17.010 ms 2024-05-27T08:45:08.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:07 smithi179 ceph-mon[36481]: pgmap v209: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-05-27T08:45:08.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:07 smithi077 ceph-mon[32007]: pgmap v209: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-05-27T08:45:08.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:07 smithi092 ceph-mon[36049]: pgmap v209: 97 pgs: 97 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 309 op/s 2024-05-27T08:45:08.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248541 4.72 MB/sec execute 182 sec latency 16.233 ms 2024-05-27T08:45:09.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248931 4.70 MB/sec execute 183 sec latency 459.211 ms 2024-05-27T08:45:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:09 smithi179 ceph-mon[36481]: pgmap v210: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-05-27T08:45:10.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:09 smithi077 ceph-mon[32007]: pgmap v210: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-05-27T08:45:10.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:09 smithi092 ceph-mon[36049]: pgmap v210: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-05-27T08:45:10.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248931 4.67 MB/sec execute 184 sec latency 1459.264 ms 2024-05-27T08:45:11.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248931 4.65 MB/sec execute 185 sec latency 2459.361 ms 2024-05-27T08:45:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:11 smithi179 ceph-mon[36481]: pgmap v211: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-05-27T08:45:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:11 smithi179 ceph-mon[36481]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T08:45:11.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:11 smithi077 ceph-mon[32007]: pgmap v211: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-05-27T08:45:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:11 smithi077 ceph-mon[32007]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T08:45:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:11 smithi092 ceph-mon[36049]: pgmap v211: 97 pgs: 97 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-05-27T08:45:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:11 smithi092 ceph-mon[36049]: osdmap e88: 12 total, 12 up, 12 in 2024-05-27T08:45:12.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248931 4.62 MB/sec execute 186 sec latency 3459.460 ms 2024-05-27T08:45:13.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248931 4.60 MB/sec execute 187 sec latency 4459.509 ms 2024-05-27T08:45:13.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:13 smithi179 ceph-mon[36481]: pgmap v213: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-05-27T08:45:13.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:13 smithi077 ceph-mon[32007]: pgmap v213: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-05-27T08:45:13.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:13 smithi092 ceph-mon[36049]: pgmap v213: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-05-27T08:45:14.605 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248931 4.57 MB/sec execute 188 sec latency 5459.559 ms 2024-05-27T08:45:15.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 248931 4.55 MB/sec execute 189 sec latency 6459.688 ms 2024-05-27T08:45:15.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:15 smithi179 ceph-mon[36481]: pgmap v214: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-05-27T08:45:15.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:15 smithi077 ceph-mon[32007]: pgmap v214: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-05-27T08:45:15.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:15 smithi092 ceph-mon[36049]: pgmap v214: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-05-27T08:45:16.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 250105 4.55 MB/sec execute 190 sec latency 6551.852 ms 2024-05-27T08:45:17.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 252197 4.60 MB/sec execute 191 sec latency 20.149 ms 2024-05-27T08:45:17.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:17 smithi179 ceph-mon[36481]: pgmap v215: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-05-27T08:45:17.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:17 smithi077 ceph-mon[32007]: pgmap v215: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-05-27T08:45:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:17 smithi092 ceph-mon[36049]: pgmap v215: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-05-27T08:45:18.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 253410 4.61 MB/sec execute 192 sec latency 18.585 ms 2024-05-27T08:45:19.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 253572 4.58 MB/sec execute 193 sec latency 889.539 ms 2024-05-27T08:45:19.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:19 smithi179 ceph-mon[36481]: pgmap v216: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.1 MiB/s wr, 150 op/s 2024-05-27T08:45:19.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:19 smithi077 ceph-mon[32007]: pgmap v216: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.1 MiB/s wr, 150 op/s 2024-05-27T08:45:19.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:19 smithi092 ceph-mon[36049]: pgmap v216: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.1 MiB/s wr, 150 op/s 2024-05-27T08:45:20.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 253572 4.56 MB/sec execute 194 sec latency 1889.577 ms 2024-05-27T08:45:21.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 255434 4.60 MB/sec execute 195 sec latency 2084.727 ms 2024-05-27T08:45:21.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:21 smithi179 ceph-mon[36481]: pgmap v217: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.1 MiB/s wr, 150 op/s 2024-05-27T08:45:21.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:21 smithi077 ceph-mon[32007]: pgmap v217: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.1 MiB/s wr, 150 op/s 2024-05-27T08:45:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:21 smithi092 ceph-mon[36049]: pgmap v217: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.1 MiB/s wr, 150 op/s 2024-05-27T08:45:22.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 256604 4.59 MB/sec execute 196 sec latency 39.538 ms 2024-05-27T08:45:23.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 257113 4.59 MB/sec execute 197 sec latency 548.725 ms 2024-05-27T08:45:23.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:23 smithi077 ceph-mon[32007]: pgmap v218: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-05-27T08:45:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:23 smithi092 ceph-mon[36049]: pgmap v218: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-05-27T08:45:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:23 smithi179 ceph-mon[36481]: pgmap v218: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 209 op/s 2024-05-27T08:45:24.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 259117 4.63 MB/sec execute 198 sec latency 640.289 ms 2024-05-27T08:45:25.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 260190 4.62 MB/sec execute 199 sec latency 23.729 ms 2024-05-27T08:45:25.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:25 smithi077 ceph-mon[32007]: pgmap v219: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:25.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:25 smithi092 ceph-mon[36049]: pgmap v219: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:25 smithi179 ceph-mon[36481]: pgmap v219: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:26.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 260661 4.61 MB/sec execute 200 sec latency 735.964 ms 2024-05-27T08:45:27.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 260661 4.59 MB/sec execute 201 sec latency 1736.051 ms 2024-05-27T08:45:27.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:27 smithi077 ceph-mon[32007]: pgmap v220: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:27 smithi092 ceph-mon[36049]: pgmap v220: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:27 smithi179 ceph-mon[36481]: pgmap v220: 97 pgs: 97 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:28.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 260661 4.57 MB/sec execute 202 sec latency 2736.120 ms 2024-05-27T08:45:28.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:28 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:45:28.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:28 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:45:28.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:28 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:45:28.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:28 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:45:28.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:28 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:45:28.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:28 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:45:28.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:28 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:45:28.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:28 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:45:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:28 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:45:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:28 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:45:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:28 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:45:29.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:28 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:45:29.549 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-05-27T08:45:29.550 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:45:29.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 260661 4.55 MB/sec execute 203 sec latency 3736.156 ms 2024-05-27T08:45:29.804 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:45:29.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:29 smithi077 ceph-mon[32007]: pgmap v221: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-05-27T08:45:29.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:29 smithi092 ceph-mon[36049]: pgmap v221: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-05-27T08:45:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:29 smithi179 ceph-mon[36481]: pgmap v221: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-05-27T08:45:30.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 260661 4.52 MB/sec execute 204 sec latency 4736.219 ms 2024-05-27T08:45:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 -- 172.21.15.77:0/455397413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd400108620 msgr2=0x7fd400108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 --2- 172.21.15.77:0/455397413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd400108620 0x7fd400108a00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009990 tx=0x7fd3f0009ca0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 -- 172.21.15.77:0/455397413 shutdown_connections 2024-05-27T08:45:31.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 --2- 172.21.15.77:0/455397413 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd4001028c0 0x7fd400110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 --2- 172.21.15.77:0/455397413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd400108620 0x7fd400108a00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 --2- 172.21.15.77:0/455397413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd400101f00 0x7fd400102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 -- 172.21.15.77:0/455397413 >> 172.21.15.77:0/455397413 conn(0x7fd400077780 msgr2=0x7fd400077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:45:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 -- 172.21.15.77:0/455397413 shutdown_connections 2024-05-27T08:45:31.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.593+0000 7fd4063f4700 1 -- 172.21.15.77:0/455397413 wait complete. 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.594+0000 7fd4063f4700 1 Processor -- start 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.594+0000 7fd4063f4700 1 -- start start 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd4063f4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd400101f00 0x7fd40019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd4063f4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4001028c0 0x7fd40019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd4063f4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd400108620 0x7fd400199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd4063f4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd400113770 con 0x7fd400108620 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd4063f4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001135f0 con 0x7fd400101f00 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd4063f4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd4001138f0 con 0x7fd4001028c0 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd400101f00 0x7fd40019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd400101f00 0x7fd40019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:32784/0 (socket says 172.21.15.77:32784) 2024-05-27T08:45:31.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 -- 172.21.15.77:0/3098856881 learned_addr learned my addr 172.21.15.77:0/3098856881 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd404991700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd400108620 0x7fd400199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3f7fff700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4001028c0 0x7fd40019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 -- 172.21.15.77:0/3098856881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4001028c0 msgr2=0x7fd40019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4001028c0 0x7fd40019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 -- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd400108620 msgr2=0x7fd400199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd400108620 0x7fd400199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 -- 172.21.15.77:0/3098856881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40019a1f0 con 0x7fd400101f00 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.595+0000 7fd3fffff700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd400101f00 0x7fd40019f8d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.596+0000 7fd3fdffb700 1 -- 172.21.15.77:0/3098856881 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0027070 con 0x7fd400101f00 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.596+0000 7fd3fdffb700 1 -- 172.21.15.77:0/3098856881 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f0022470 con 0x7fd400101f00 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.596+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40019a480 con 0x7fd400101f00 2024-05-27T08:45:31.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.596+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4001a67a0 con 0x7fd400101f00 2024-05-27T08:45:31.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.596+0000 7fd3fdffb700 1 -- 172.21.15.77:0/3098856881 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0017650 con 0x7fd400101f00 2024-05-27T08:45:31.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.597+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40006a230 con 0x7fd400101f00 2024-05-27T08:45:31.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.598+0000 7fd3fdffb700 1 -- 172.21.15.77:0/3098856881 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd3f002a070 con 0x7fd400101f00 2024-05-27T08:45:31.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.599+0000 7fd3fdffb700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd3ec0b29f0 0x7fd3ec0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:31.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.599+0000 7fd3f7fff700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd3ec0b29f0 0x7fd3ec0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:31.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.599+0000 7fd3fdffb700 1 -- 172.21.15.77:0/3098856881 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fd3f00e3380 con 0x7fd400101f00 2024-05-27T08:45:31.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.599+0000 7fd3f7fff700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd3ec0b29f0 0x7fd3ec0b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd400077d80 tx=0x7fd3e8009260 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:45:31.604 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.604+0000 7fd3fdffb700 1 -- 172.21.15.77:0/3098856881 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3f0013070 con 0x7fd400101f00 2024-05-27T08:45:31.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 262652 4.56 MB/sec execute 205 sec latency 4810.028 ms 2024-05-27T08:45:31.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:31 smithi077 ceph-mon[32007]: pgmap v222: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-27T08:45:31.886 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.886+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd40002e560 con 0x7fd400101f00 2024-05-27T08:45:31.887 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.887+0000 7fd3fdffb700 1 -- 172.21.15.77:0/3098856881 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11934 (secure 0 0 0) 0x7fd3f00aade0 con 0x7fd400101f00 2024-05-27T08:45:31.887 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:45:31.887 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:44:43.671254+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:45:31.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.889+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd3ec0b29f0 msgr2=0x7fd3ec0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:31.889 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.889+0000 7fd4063f4700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd3ec0b29f0 0x7fd3ec0b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd400077d80 tx=0x7fd3e8009260 comp rx=0 tx=0).stop 2024-05-27T08:45:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.889+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd400101f00 msgr2=0x7fd40019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.889+0000 7fd4063f4700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd400101f00 0x7fd40019f8d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f000bfd0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.890+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 shutdown_connections 2024-05-27T08:45:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.890+0000 7fd4063f4700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fd3ec0b29f0 0x7fd3ec0b4eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.890 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.890+0000 7fd4063f4700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd400108620 0x7fd400199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.890+0000 7fd4063f4700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd400101f00 0x7fd40019f8d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.890+0000 7fd4063f4700 1 --2- 172.21.15.77:0/3098856881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4001028c0 0x7fd40019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.890+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 >> 172.21.15.77:0/3098856881 conn(0x7fd400077780 msgr2=0x7fd400100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:45:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.890+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 shutdown_connections 2024-05-27T08:45:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:31.890+0000 7fd4063f4700 1 -- 172.21.15.77:0/3098856881 wait complete. 2024-05-27T08:45:31.891 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 37 2024-05-27T08:45:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:31 smithi092 ceph-mon[36049]: pgmap v222: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-27T08:45:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:31 smithi179 ceph-mon[36481]: pgmap v222: 97 pgs: 97 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-05-27T08:45:32.551 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-05-27T08:45:32.552 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-05-27T08:45:32.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 263915 4.57 MB/sec execute 206 sec latency 18.085 ms 2024-05-27T08:45:32.815 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:45:32.842 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:32 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3098856881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:45:32.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:32 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3098856881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:45:33.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:32 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3098856881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:45:33.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264204 4.55 MB/sec execute 207 sec latency 670.035 ms 2024-05-27T08:45:33.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:33 smithi077 ceph-mon[32007]: pgmap v223: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-05-27T08:45:33.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:33 smithi092 ceph-mon[36049]: pgmap v223: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-05-27T08:45:34.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:33 smithi179 ceph-mon[36481]: pgmap v223: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-05-27T08:45:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 -- 172.21.15.77:0/2435947113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca20102440 msgr2=0x7fca201028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.390 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 --2- 172.21.15.77:0/2435947113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca20102440 0x7fca201028c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fca08009990 tx=0x7fca08009ca0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 -- 172.21.15.77:0/2435947113 shutdown_connections 2024-05-27T08:45:34.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 --2- 172.21.15.77:0/2435947113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca20102e00 0x7fca2010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 --2- 172.21.15.77:0/2435947113 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca201084c0 0x7fca201088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 --2- 172.21.15.77:0/2435947113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca20102440 0x7fca201028c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 -- 172.21.15.77:0/2435947113 >> 172.21.15.77:0/2435947113 conn(0x7fca200793e0 msgr2=0x7fca200797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:45:34.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 -- 172.21.15.77:0/2435947113 shutdown_connections 2024-05-27T08:45:34.391 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 -- 172.21.15.77:0/2435947113 wait complete. 2024-05-27T08:45:34.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 Processor -- start 2024-05-27T08:45:34.392 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.390+0000 7fca26025700 1 -- start start 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca26025700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca20102440 0x7fca20104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca26025700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca20102e00 0x7fca201045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca26025700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca201084c0 0x7fca20104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca26025700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca201126d0 con 0x7fca201084c0 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca26025700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca20112550 con 0x7fca20102440 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca26025700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca20112850 con 0x7fca20102e00 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca20102440 0x7fca20104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1f7fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca20102440 0x7fca20104090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.77:32804/0 (socket says 172.21.15.77:32804) 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1f7fe700 1 -- 172.21.15.77:0/1888043494 learned_addr learned my addr 172.21.15.77:0/1888043494 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca17fff700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca20102e00 0x7fca201045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1ffff700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca201084c0 0x7fca20104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1f7fe700 1 -- 172.21.15.77:0/1888043494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca20102e00 msgr2=0x7fca201045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.393 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca20102e00 0x7fca201045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1f7fe700 1 -- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca201084c0 msgr2=0x7fca20104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca201084c0 0x7fca20104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1f7fe700 1 -- 172.21.15.77:0/1888043494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2019ddf0 con 0x7fca20102440 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.391+0000 7fca1ffff700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca201084c0 0x7fca20104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.392+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca20102440 0x7fca20104090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca1000b530 tx=0x7fca1000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.392+0000 7fca1d7fa700 1 -- 172.21.15.77:0/1888043494 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10015020 con 0x7fca20102440 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.392+0000 7fca26025700 1 -- 172.21.15.77:0/1888043494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2019e0e0 con 0x7fca20102440 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.392+0000 7fca1d7fa700 1 -- 172.21.15.77:0/1888043494 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca10011ce0 con 0x7fca20102440 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.392+0000 7fca1d7fa700 1 -- 172.21.15.77:0/1888043494 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10010dd0 con 0x7fca20102440 2024-05-27T08:45:34.394 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.392+0000 7fca26025700 1 -- 172.21.15.77:0/1888043494 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2019e640 con 0x7fca20102440 2024-05-27T08:45:34.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.395+0000 7fca1d7fa700 1 -- 172.21.15.77:0/1888043494 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fca10018020 con 0x7fca20102440 2024-05-27T08:45:34.395 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.395+0000 7fca1d7fa700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fca0c0b2aa0 0x7fca0c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.395+0000 7fca17fff700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fca0c0b2aa0 0x7fca0c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.395+0000 7fca1d7fa700 1 -- 172.21.15.77:0/1888043494 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fca100e1be0 con 0x7fca20102440 2024-05-27T08:45:34.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca308bbb80 1 Processor -- start 2024-05-27T08:45:34.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca308bbb80 1 -- start start 2024-05-27T08:45:34.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57ce5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57ce1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.397 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce1640 0x562d57ced6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca308bbb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d57c3ef00 con 0x562d57ce0c80 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca308bbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562d57c2eaa0 con 0x562d57ce1640 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca308bbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562d57c32ae0 con 0x562d57ce57b0 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fca0c0b2aa0 0x7fca0c0b4f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fca2019ddc0 tx=0x7fca08005960 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57ce1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57ce1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33362/0 (socket says 172.21.15.77:33362) 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 -- 172.21.15.77:0/1072750981 learned_addr learned my addr 172.21.15.77:0/1072750981 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca1ffff700 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce1640 0x562d57ced6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57ce5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 -- 172.21.15.77:0/1072750981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 msgr2=0x562d57ce5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57ce5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.398 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 -- 172.21.15.77:0/1072750981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce1640 msgr2=0x562d57ced6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:45:34.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce1640 0x562d57ced6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 -- 172.21.15.77:0/1072750981 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d57bd8140 con 0x562d57ce0c80 2024-05-27T08:45:34.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.396+0000 7fca17fff700 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57ce1100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fca08015000 tx=0x7fca0801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca85a6edef116b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:45:34.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca15ffb700 1 -- 172.21.15.77:0/1072750981 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08034070 con 0x562d57ce0c80 2024-05-27T08:45:34.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca15ffb700 1 -- 172.21.15.77:0/1072750981 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca080249f0 con 0x562d57ce0c80 2024-05-27T08:45:34.399 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca15ffb700 1 -- 172.21.15.77:0/1072750981 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0802f9f0 con 0x562d57ce0c80 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca308bbb80 1 -- 172.21.15.77:0/1072750981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 msgr2=0x562d57ce1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca308bbb80 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57ce1100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fca08015000 tx=0x7fca0801f000 comp rx=0 tx=0).stop 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca308bbb80 1 -- 172.21.15.77:0/1072750981 shutdown_connections 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca308bbb80 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce1640 0x562d57ced6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca308bbb80 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57ce5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca308bbb80 1 --2- 172.21.15.77:0/1072750981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57ce1100 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.397+0000 7fca308bbb80 1 -- 172.21.15.77:0/1072750981 >> 172.21.15.77:0/1072750981 conn(0x562d57c0c3d0 msgr2=0x562d57cdb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 -- 172.21.15.77:0/1072750981 shutdown_connections 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 -- 172.21.15.77:0/1072750981 wait complete. 2024-05-27T08:45:34.403 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 -- start start 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 Processor -- start 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce0c80 0x562d57ced350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 0x562d57ceab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d57b35130 con 0x562d57ce1640 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x562d57b96f30 con 0x562d57ce57b0 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.398+0000 7fca308bbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562d57cedcb0 con 0x562d57ce0c80 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca308bbb80 10 client.0 _ll_register_callbacks cb 0x562d57bc9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33368/0 (socket says 172.21.15.77:33368) 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 -- 172.21.15.77:0/2838725185 learned_addr learned my addr 172.21.15.77:0/2838725185 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca1ffff700 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 0x562d57ceab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.404 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce0c80 0x562d57ced350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 -- 172.21.15.77:0/2838725185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce0c80 msgr2=0x562d57ced350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce0c80 0x562d57ced350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 -- 172.21.15.77:0/2838725185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 msgr2=0x562d57ceab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 0x562d57ceab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 -- 172.21.15.77:0/2838725185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d57b7ea60 con 0x562d57ce1640 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca17fff700 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cea650 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fca0800b640 tx=0x7fca080247f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729f70c5b8cda90e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fc9feffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fc9feffd700 1 -- 172.21.15.77:0/2838725185 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08034070 con 0x562d57ce1640 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca308bbb80 1 -- 172.21.15.77:0/2838725185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca080249f0 con 0x562d57ce1640 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fc9feffd700 1 -- 172.21.15.77:0/2838725185 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fca0802f800 con 0x562d57ce1640 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fc9feffd700 1 -- 172.21.15.77:0/2838725185 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fca0802fa00 con 0x562d57ce1640 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca308bbb80 10 client.15693 fetch_fsmap learned FSMap version 37 2024-05-27T08:45:34.405 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.399+0000 7fca308bbb80 1 -- 172.21.15.77:0/2838725185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562d57be88f0 con 0x562d57ce1640 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fc9feffd700 1 -- 172.21.15.77:0/2838725185 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca080249f0 con 0x562d57ce1640 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fc9feffd700 1 -- 172.21.15.77:0/2838725185 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fca08041420 con 0x562d57ce1640 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fca308bbb80 10 client.15693 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fca308bbb80 10 client.15693 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fca308bbb80 20 client.15693 populate_metadata read hostname 'smithi077' 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x562d57cfa6c0 0x562d57cfcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fca308bbb80 4 client.15693 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fca308bbb80 1 -- 172.21.15.77:0/2838725185 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d57c40e50 con 0x562d57cfa6c0 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.400+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x562d57cfa6c0 0x562d57cfcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.401+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x562d57cfa6c0 0x562d57cfcab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b65da00e9ffa77f1 server_cookie=66a9ddab198a45fb in_seq=0 out_seq=0 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.401+0000 7fc9feffd700 10 client.15693 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:45:34.406 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.401+0000 7fc9feffd700 1 -- 172.21.15.77:0/2838725185 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562d57c40e50 con 0x562d57cfa6c0 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.401+0000 7fc9feffd700 10 client.15693 handle_command_reply: tid=0 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.401+0000 7fca308bbb80 1 client.15693 shutdown 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 -- 172.21.15.77:0/2838725185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 msgr2=0x562d57cea650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cea650 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fca0800b640 tx=0x7fca080247f0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 -- 172.21.15.77:0/2838725185 shutdown_connections 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x562d57cfa6c0 0x562d57cfcab0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 0x562d57ceab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fc9feffd700 0 client.15693 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce0c80 0x562d57ced350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2838725185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cea650 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 -- 172.21.15.77:0/2838725185 >> 172.21.15.77:0/2838725185 conn(0x562d57c0c3d0 msgr2=0x562d57cdfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 -- 172.21.15.77:0/2838725185 shutdown_connections 2024-05-27T08:45:34.407 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 1 -- 172.21.15.77:0/2838725185 wait complete. 2024-05-27T08:45:34.408 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.402+0000 7fca308bbb80 20 client.15693 trim_cache size 0 max 16384 2024-05-27T08:45:34.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca308bbb80 1 Processor -- start 2024-05-27T08:45:34.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca308bbb80 1 -- start start 2024-05-27T08:45:34.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57cd5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce1640 0x562d57ce97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 0x562d57ce6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca308bbb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca201a5bf0 con 0x562d57ce0c80 2024-05-27T08:45:34.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca308bbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca201be830 con 0x562d57ce57b0 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca308bbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca201a62c0 con 0x562d57ce1640 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57cd5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57cd5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33378/0 (socket says 172.21.15.77:33378) 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 -- 172.21.15.77:0/3272485181 learned_addr learned my addr 172.21.15.77:0/3272485181 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1ffff700 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 0x562d57ce6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 -- 172.21.15.77:0/3272485181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce1640 msgr2=0x562d57ce97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca17fff700 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce1640 0x562d57ce97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce1640 0x562d57ce97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 -- 172.21.15.77:0/3272485181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 msgr2=0x562d57ce6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 0x562d57ce6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 -- 172.21.15.77:0/3272485181 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d57cdd080 con 0x562d57ce0c80 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.422+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57cd5f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fca100a62e0 tx=0x7fca10010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edf3220232d8d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:45:34.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca157fa700 1 -- 172.21.15.77:0/3272485181 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca100bc020 con 0x562d57ce0c80 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca157fa700 1 -- 172.21.15.77:0/3272485181 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca100b1e20 con 0x562d57ce0c80 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca157fa700 1 -- 172.21.15.77:0/3272485181 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca100b1bf0 con 0x562d57ce0c80 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 -- 172.21.15.77:0/3272485181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 msgr2=0x562d57cd5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57cd5f00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fca100a62e0 tx=0x7fca10010870 comp rx=0 tx=0).stop 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 -- 172.21.15.77:0/3272485181 shutdown_connections 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce57b0 0x562d57ce6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce0c80 0x562d57cd5f00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 --2- 172.21.15.77:0/3272485181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce1640 0x562d57ce97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 -- 172.21.15.77:0/3272485181 >> 172.21.15.77:0/3272485181 conn(0x562d57c0c3d0 msgr2=0x562d57ce6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 -- 172.21.15.77:0/3272485181 shutdown_connections 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.423+0000 7fca308bbb80 1 -- 172.21.15.77:0/3272485181 wait complete. 2024-05-27T08:45:34.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 1 -- start start 2024-05-27T08:45:34.427 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 1 Processor -- start 2024-05-27T08:45:34.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce0c80 0x562d57cfbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cfc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57d63ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca201be0a0 con 0x562d57ce1640 2024-05-27T08:45:34.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fca201bd9d0 con 0x562d57ce0c80 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca201a3bb0 con 0x562d57ce57b0 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca308bbb80 10 client.0 _ll_register_callbacks cb 0x562d57bc9ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca17fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cfc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca17fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cfc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:33380/0 (socket says 172.21.15.77:33380) 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca17fff700 1 -- 172.21.15.77:0/2746071907 learned_addr learned my addr 172.21.15.77:0/2746071907 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.424+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce0c80 0x562d57cfbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca1ffff700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57d63ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca17fff700 1 -- 172.21.15.77:0/2746071907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 msgr2=0x562d57d63ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca17fff700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57d63ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca17fff700 1 -- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce0c80 msgr2=0x562d57cfbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca17fff700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce0c80 0x562d57cfbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca17fff700 1 -- 172.21.15.77:0/2746071907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d57d64530 con 0x562d57ce1640 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca1ffff700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57d63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce0c80 0x562d57cfbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca17fff700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cfc4e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca08009750 tx=0x7fca0800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e66ddb22d2f3068 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fc9feffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fc9feffd700 1 -- 172.21.15.77:0/2746071907 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08034040 con 0x562d57ce1640 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fc9feffd700 1 -- 172.21.15.77:0/2746071907 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7fca0800f460 con 0x562d57ce1640 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca308bbb80 1 -- 172.21.15.77:0/2746071907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca100b1e20 con 0x562d57ce1640 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fc9feffd700 1 -- 172.21.15.77:0/2746071907 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fca08041c70 con 0x562d57ce1640 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca308bbb80 10 client.15699 fetch_fsmap learned FSMap version 37 2024-05-27T08:45:34.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fca308bbb80 1 -- 172.21.15.77:0/2746071907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562d57d657c0 con 0x562d57ce1640 2024-05-27T08:45:34.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fc9feffd700 1 -- 172.21.15.77:0/2746071907 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca100b1e20 con 0x562d57ce1640 2024-05-27T08:45:34.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.425+0000 7fc9feffd700 1 -- 172.21.15.77:0/2746071907 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fca0804a590 con 0x562d57ce1640 2024-05-27T08:45:34.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fca308bbb80 10 client.15699 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:45:34.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fca308bbb80 10 client.15699 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:45:34.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fca308bbb80 20 client.15699 populate_metadata read hostname 'smithi077' 2024-05-27T08:45:34.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x562d57d65df0 0x562d57d68210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:45:34.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fca308bbb80 4 client.15699 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-05-27T08:45:34.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fca308bbb80 1 -- 172.21.15.77:0/2746071907 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x562d57c3a630 con 0x562d57d65df0 2024-05-27T08:45:34.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x562d57d65df0 0x562d57d68210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:45:34.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fc9feffd700 10 client.15699 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:45:34.433 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.426+0000 7fca1f7fe700 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x562d57d65df0 0x562d57d68210 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a90fbd93ef47ad server_cookie=7b86b1cda1ed9680 in_seq=0 out_seq=0 2024-05-27T08:45:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.500+0000 7fc9feffd700 1 -- 172.21.15.77:0/2746071907 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562d57c3a630 con 0x562d57d65df0 2024-05-27T08:45:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.500+0000 7fc9feffd700 10 client.15699 handle_command_reply: tid=0 2024-05-27T08:45:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.500+0000 7fca308bbb80 1 client.15699 shutdown 2024-05-27T08:45:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 -- 172.21.15.77:0/2746071907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 msgr2=0x562d57cfc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cfc4e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca08009750 tx=0x7fca0800b760 comp rx=0 tx=0).stop 2024-05-27T08:45:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 -- 172.21.15.77:0/2746071907 shutdown_connections 2024-05-27T08:45:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x562d57d65df0 0x562d57d68210 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562d57ce57b0 0x562d57d63ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562d57ce0c80 0x562d57cfbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 --2- 172.21.15.77:0/2746071907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d57ce1640 0x562d57cfc4e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 -- 172.21.15.77:0/2746071907 >> 172.21.15.77:0/2746071907 conn(0x562d57c0c3d0 msgr2=0x562d57cea790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fc9feffd700 0 client.15699 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 -- 172.21.15.77:0/2746071907 shutdown_connections 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 1 -- 172.21.15.77:0/2746071907 wait complete. 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.501+0000 7fca308bbb80 20 client.15699 trim_cache size 0 max 16384 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 37, 2024-05-27T08:45:34.503 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 22, 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "cc9d5d70": { 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 22, 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 149.8, 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:45:34.504 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:45:34.505 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:45:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 -- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fca0c0b2aa0 msgr2=0x7fca0c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fca0c0b2aa0 0x7fca0c0b4f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fca2019ddc0 tx=0x7fca08005960 comp rx=0 tx=0).stop 2024-05-27T08:45:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 -- 172.21.15.77:0/1888043494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca20102440 msgr2=0x7fca20104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:45:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca20102440 0x7fca20104090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca1000b530 tx=0x7fca1000b840 comp rx=0 tx=0).stop 2024-05-27T08:45:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 -- 172.21.15.77:0/1888043494 shutdown_connections 2024-05-27T08:45:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fca0c0b2aa0 0x7fca0c0b4f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca201084c0 0x7fca20104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.506 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fca20102440 0x7fca20104090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 --2- 172.21.15.77:0/1888043494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca20102e00 0x7fca201045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:45:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 -- 172.21.15.77:0/1888043494 >> 172.21.15.77:0/1888043494 conn(0x7fca200793e0 msgr2=0x7fca2006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:45:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 -- 172.21.15.77:0/1888043494 shutdown_connections 2024-05-27T08:45:34.507 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:45:34.502+0000 7fca26025700 1 -- 172.21.15.77:0/1888043494 wait complete. 2024-05-27T08:45:34.606 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.53 MB/sec execute 208 sec latency 1510.776 ms 2024-05-27T08:45:35.232 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cc9d5d70', quiesce took 5.7 seconds. Will release after: 24.1 2024-05-27T08:45:35.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.51 MB/sec execute 209 sec latency 1159.348 ms 2024-05-27T08:45:35.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:35 smithi092 ceph-mon[36049]: pgmap v224: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:36.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:35 smithi179 ceph-mon[36481]: pgmap v224: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:36.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:35 smithi077 ceph-mon[32007]: pgmap v224: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:36.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.48 MB/sec execute 210 sec latency 2159.400 ms 2024-05-27T08:45:37.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.46 MB/sec execute 211 sec latency 3159.450 ms 2024-05-27T08:45:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:37 smithi092 ceph-mon[36049]: pgmap v225: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:38.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:37 smithi179 ceph-mon[36481]: pgmap v225: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:38.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:37 smithi077 ceph-mon[32007]: pgmap v225: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-05-27T08:45:38.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.44 MB/sec execute 212 sec latency 4159.554 ms 2024-05-27T08:45:39.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.42 MB/sec execute 213 sec latency 5159.669 ms 2024-05-27T08:45:39.907 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:39 smithi179 ceph-mon[36481]: pgmap v226: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-05-27T08:45:39.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:39 smithi092 ceph-mon[36049]: pgmap v226: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-05-27T08:45:40.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:39 smithi077 ceph-mon[32007]: pgmap v226: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-05-27T08:45:40.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.40 MB/sec execute 214 sec latency 6159.757 ms 2024-05-27T08:45:41.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.38 MB/sec execute 215 sec latency 7159.850 ms 2024-05-27T08:45:41.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:41 smithi092 ceph-mon[36049]: pgmap v227: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-27T08:45:42.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:41 smithi179 ceph-mon[36481]: pgmap v227: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-27T08:45:42.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:41 smithi077 ceph-mon[32007]: pgmap v227: 97 pgs: 97 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 1.4 MiB/s wr, 95 op/s 2024-05-27T08:45:42.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.36 MB/sec execute 216 sec latency 8159.916 ms 2024-05-27T08:45:43.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.34 MB/sec execute 217 sec latency 9159.959 ms 2024-05-27T08:45:43.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:43 smithi092 ceph-mon[36049]: pgmap v228: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-05-27T08:45:44.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:43 smithi179 ceph-mon[36481]: pgmap v228: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-05-27T08:45:44.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:43 smithi077 ceph-mon[32007]: pgmap v228: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-05-27T08:45:44.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.32 MB/sec execute 218 sec latency 10160.081 ms 2024-05-27T08:45:45.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.30 MB/sec execute 219 sec latency 11160.207 ms 2024-05-27T08:45:45.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:45 smithi092 ceph-mon[36049]: pgmap v229: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 64 op/s 2024-05-27T08:45:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:45 smithi179 ceph-mon[36481]: pgmap v229: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 64 op/s 2024-05-27T08:45:46.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:45 smithi077 ceph-mon[32007]: pgmap v229: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 64 op/s 2024-05-27T08:45:46.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.28 MB/sec execute 220 sec latency 12160.287 ms 2024-05-27T08:45:47.607 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.26 MB/sec execute 221 sec latency 13160.349 ms 2024-05-27T08:45:47.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:47 smithi092 ceph-mon[36049]: pgmap v230: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 64 op/s 2024-05-27T08:45:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:47 smithi179 ceph-mon[36481]: pgmap v230: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 64 op/s 2024-05-27T08:45:48.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:47 smithi077 ceph-mon[32007]: pgmap v230: 97 pgs: 97 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.2 MiB/s wr, 64 op/s 2024-05-27T08:45:48.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.24 MB/sec execute 222 sec latency 14160.437 ms 2024-05-27T08:45:49.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.22 MB/sec execute 223 sec latency 15160.544 ms 2024-05-27T08:45:49.931 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:49 smithi077 ceph-mon[32007]: pgmap v231: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-05-27T08:45:49.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:49 smithi092 ceph-mon[36049]: pgmap v231: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-05-27T08:45:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:49 smithi179 ceph-mon[36481]: pgmap v231: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-05-27T08:45:50.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.20 MB/sec execute 224 sec latency 16160.580 ms 2024-05-27T08:45:51.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.19 MB/sec execute 225 sec latency 17160.658 ms 2024-05-27T08:45:51.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:51 smithi092 ceph-mon[36049]: pgmap v232: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 617 KiB/s wr, 35 op/s 2024-05-27T08:45:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:51 smithi179 ceph-mon[36481]: pgmap v232: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 617 KiB/s wr, 35 op/s 2024-05-27T08:45:52.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:51 smithi077 ceph-mon[32007]: pgmap v232: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 617 KiB/s wr, 35 op/s 2024-05-27T08:45:52.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.17 MB/sec execute 226 sec latency 18160.742 ms 2024-05-27T08:45:53.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.15 MB/sec execute 227 sec latency 19160.850 ms 2024-05-27T08:45:53.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:53 smithi092 ceph-mon[36049]: pgmap v233: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 881 KiB/s wr, 49 op/s 2024-05-27T08:45:54.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:53 smithi179 ceph-mon[36481]: pgmap v233: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 881 KiB/s wr, 49 op/s 2024-05-27T08:45:54.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:53 smithi077 ceph-mon[32007]: pgmap v233: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 881 KiB/s wr, 49 op/s 2024-05-27T08:45:54.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.13 MB/sec execute 228 sec latency 20160.966 ms 2024-05-27T08:45:55.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.11 MB/sec execute 229 sec latency 21161.027 ms 2024-05-27T08:45:55.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:55 smithi092 ceph-mon[36049]: pgmap v234: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 293 KiB/s wr, 25 op/s 2024-05-27T08:45:56.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:55 smithi179 ceph-mon[36481]: pgmap v234: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 293 KiB/s wr, 25 op/s 2024-05-27T08:45:56.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:55 smithi077 ceph-mon[32007]: pgmap v234: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 293 KiB/s wr, 25 op/s 2024-05-27T08:45:56.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.09 MB/sec execute 230 sec latency 22161.074 ms 2024-05-27T08:45:57.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.08 MB/sec execute 231 sec latency 23161.161 ms 2024-05-27T08:45:57.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:57 smithi092 ceph-mon[36049]: pgmap v235: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 293 KiB/s wr, 25 op/s 2024-05-27T08:45:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:57 smithi179 ceph-mon[36481]: pgmap v235: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 293 KiB/s wr, 25 op/s 2024-05-27T08:45:58.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:57 smithi077 ceph-mon[32007]: pgmap v235: 97 pgs: 97 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 293 KiB/s wr, 25 op/s 2024-05-27T08:45:58.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.06 MB/sec execute 232 sec latency 24161.255 ms 2024-05-27T08:45:59.334 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:45:59.593 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:45:59.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.04 MB/sec execute 233 sec latency 25161.300 ms 2024-05-27T08:45:59.969 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:45:59 smithi077 ceph-mon[32007]: pgmap v236: 97 pgs: 97 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 788 KiB/s wr, 47 op/s 2024-05-27T08:45:59.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:45:59 smithi092 ceph-mon[36049]: pgmap v236: 97 pgs: 97 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 788 KiB/s wr, 47 op/s 2024-05-27T08:46:00.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:45:59 smithi179 ceph-mon[36481]: pgmap v236: 97 pgs: 97 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 788 KiB/s wr, 47 op/s 2024-05-27T08:46:00.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.02 MB/sec execute 234 sec latency 26161.341 ms 2024-05-27T08:46:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 -- 172.21.15.77:0/396543034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c103f90 msgr2=0x7f9c5c110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 --2- 172.21.15.77:0/396543034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c103f90 0x7f9c5c110440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400b0d0 tx=0x7f9c5400b3e0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 -- 172.21.15.77:0/396543034 shutdown_connections 2024-05-27T08:46:01.419 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 --2- 172.21.15.77:0/396543034 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c103f90 0x7f9c5c110440 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 --2- 172.21.15.77:0/396543034 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c109670 0x7f9c5c109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 --2- 172.21.15.77:0/396543034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c5c1035d0 0x7f9c5c103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 -- 172.21.15.77:0/396543034 >> 172.21.15.77:0/396543034 conn(0x7f9c5c07a730 msgr2=0x7f9c5c07ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 -- 172.21.15.77:0/396543034 shutdown_connections 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.416+0000 7f9c60f09700 1 -- 172.21.15.77:0/396543034 wait complete. 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.417+0000 7f9c60f09700 1 Processor -- start 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.417+0000 7f9c60f09700 1 -- start start 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c60f09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c5c1035d0 0x7f9c5c19f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c60f09700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c103f90 0x7f9c5c19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c60f09700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c109670 0x7f9c5c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c60f09700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c1137d0 con 0x7f9c5c109670 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c60f09700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c113650 con 0x7f9c5c103f90 2024-05-27T08:46:01.420 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c60f09700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c113950 con 0x7f9c5c1035d0 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c109670 0x7f9c5c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c109670 0x7f9c5c199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60802/0 (socket says 172.21.15.77:60802) 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 -- 172.21.15.77:0/2309368930 learned_addr learned my addr 172.21.15.77:0/2309368930 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c59d9b700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c103f90 0x7f9c5c19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5a59c700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c5c1035d0 0x7f9c5c19f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 -- 172.21.15.77:0/2309368930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c5c1035d0 msgr2=0x7f9c5c19f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c5c1035d0 0x7f9c5c19f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 -- 172.21.15.77:0/2309368930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c103f90 msgr2=0x7f9c5c19fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c103f90 0x7f9c5c19fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 -- 172.21.15.77:0/2309368930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c19a270 con 0x7f9c5c109670 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5a59c700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c5c1035d0 0x7f9c5c19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c59d9b700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c103f90 0x7f9c5c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:46:01.421 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.418+0000 7f9c5ad9d700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c109670 0x7f9c5c199b10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9c540095c0 tx=0x7f9c54007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:46:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.419+0000 7f9c4b7fe700 1 -- 172.21.15.77:0/2309368930 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5401f070 con 0x7f9c5c109670 2024-05-27T08:46:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.419+0000 7f9c4b7fe700 1 -- 172.21.15.77:0/2309368930 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c54003c00 con 0x7f9c5c109670 2024-05-27T08:46:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.419+0000 7f9c4b7fe700 1 -- 172.21.15.77:0/2309368930 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5401b6a0 con 0x7f9c5c109670 2024-05-27T08:46:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.419+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c19a500 con 0x7f9c5c109670 2024-05-27T08:46:01.422 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.419+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c1a67d0 con 0x7f9c5c109670 2024-05-27T08:46:01.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.420+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c104ce0 con 0x7f9c5c109670 2024-05-27T08:46:01.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.422+0000 7f9c4b7fe700 1 -- 172.21.15.77:0/2309368930 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9c54022070 con 0x7f9c5c109670 2024-05-27T08:46:01.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.423+0000 7f9c4b7fe700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c440b29f0 0x7f9c440b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:01.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.423+0000 7f9c5a59c700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c440b29f0 0x7f9c440b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:01.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.423+0000 7f9c4b7fe700 1 -- 172.21.15.77:0/2309368930 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8542+0+0 (secure 0 0 0) 0x7f9c5400f020 con 0x7f9c5c109670 2024-05-27T08:46:01.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.423+0000 7f9c5a59c700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c440b29f0 0x7f9c440b4eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c005fd0 tx=0x7f9c4c005f60 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:46:01.426 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.426+0000 7f9c4b7fe700 1 -- 172.21.15.77:0/2309368930 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c54029020 con 0x7f9c5c109670 2024-05-27T08:46:01.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 4.01 MB/sec execute 235 sec latency 27161.397 ms 2024-05-27T08:46:01.711 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.711+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c5c19aff0 con 0x7f9c5c109670 2024-05-27T08:46:01.712 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.712+0000 7f9c4b7fe700 1 -- 172.21.15.77:0/2309368930 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11934 (secure 0 0 0) 0x7f9c540ab6e0 con 0x7f9c5c109670 2024-05-27T08:46:01.712 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:46:01.713 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:44:43.671254+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c440b29f0 msgr2=0x7f9c440b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c440b29f0 0x7f9c440b4eb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c005fd0 tx=0x7f9c4c005f60 comp rx=0 tx=0).stop 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c109670 msgr2=0x7f9c5c199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c109670 0x7f9c5c199b10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9c540095c0 tx=0x7f9c54007a20 comp rx=0 tx=0).stop 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 shutdown_connections 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9c440b29f0 0x7f9c440b4eb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c5c109670 0x7f9c5c199b10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c5c1035d0 0x7f9c5c19f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 --2- 172.21.15.77:0/2309368930 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9c5c103f90 0x7f9c5c19fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:01.715 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 >> 172.21.15.77:0/2309368930 conn(0x7f9c5c07a730 msgr2=0x7f9c5c1021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:46:01.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 shutdown_connections 2024-05-27T08:46:01.716 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:01.714+0000 7f9c60f09700 1 -- 172.21.15.77:0/2309368930 wait complete. 2024-05-27T08:46:01.716 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 37 2024-05-27T08:46:01.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:01 smithi077 ceph-mon[32007]: pgmap v237: 97 pgs: 97 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 760 KiB/s wr, 36 op/s 2024-05-27T08:46:01.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:01 smithi092 ceph-mon[36049]: pgmap v237: 97 pgs: 97 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 760 KiB/s wr, 36 op/s 2024-05-27T08:46:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:01 smithi179 ceph-mon[36481]: pgmap v237: 97 pgs: 97 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 760 KiB/s wr, 36 op/s 2024-05-27T08:46:02.346 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id cc9d5d70 --release --await' 2024-05-27T08:46:02.346 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db --set-id cc9d5d70 --release --await 2024-05-27T08:46:02.594 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:46:02.608 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 3.99 MB/sec execute 236 sec latency 28161.423 ms 2024-05-27T08:46:02.962 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:02 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2309368930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:46:02.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:02 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2309368930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:46:03.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:02 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2309368930' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:46:03.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264207 3.97 MB/sec execute 237 sec latency 29161.453 ms 2024-05-27T08:46:03.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:03 smithi092 ceph-mon[36049]: pgmap v238: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 915 KiB/s wr, 72 op/s 2024-05-27T08:46:03.988 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:03 smithi077 ceph-mon[32007]: pgmap v238: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 915 KiB/s wr, 72 op/s 2024-05-27T08:46:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:03 smithi179 ceph-mon[36481]: pgmap v238: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 915 KiB/s wr, 72 op/s 2024-05-27T08:46:04.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3616689970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec1095b0 msgr2=0x7fb7ec109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3616689970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec1095b0 0x7fb7ec109990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009990 tx=0x7fb7e8009ca0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3616689970 shutdown_connections 2024-05-27T08:46:04.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3616689970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ec06c650 0x7fb7ec1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.198 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3616689970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec1095b0 0x7fb7ec109990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3616689970 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ec06bc90 0x7fb7ec06c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3616689970 >> 172.21.15.77:0/3616689970 conn(0x7fb7ec079410 msgr2=0x7fb7ec079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3616689970 shutdown_connections 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3616689970 wait complete. 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.196+0000 7fb7f3e80700 1 Processor -- start 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f3e80700 1 -- start start 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f3e80700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ec06bc90 0x7fb7ec199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f3e80700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec06c650 0x7fb7ec19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f3e80700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ec1095b0 0x7fb7ec19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f3e80700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec113740 con 0x7fb7ec06c650 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f3e80700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1135c0 con 0x7fb7ec06bc90 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f3e80700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1138c0 con 0x7fb7ec1095b0 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec06c650 0x7fb7ec19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.199 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec06c650 0x7fb7ec19a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60828/0 (socket says 172.21.15.77:60828) 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 -- 172.21.15.77:0/3709381204 learned_addr learned my addr 172.21.15.77:0/3709381204 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ec06bc90 0x7fb7ec199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f241d700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ec1095b0 0x7fb7ec19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 -- 172.21.15.77:0/3709381204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ec1095b0 msgr2=0x7fb7ec19e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ec1095b0 0x7fb7ec19e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 -- 172.21.15.77:0/3709381204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ec06bc90 msgr2=0x7fb7ec199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ec06bc90 0x7fb7ec199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 -- 172.21.15.77:0/3709381204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec19ed20 con 0x7fb7ec06c650 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.197+0000 7fb7f141b700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec06c650 0x7fb7ec19a250 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c920 tx=0x7fb7dc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.198+0000 7fb7e2ffd700 1 -- 172.21.15.77:0/3709381204 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc007b00 con 0x7fb7ec06c650 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.198+0000 7fb7e2ffd700 1 -- 172.21.15.77:0/3709381204 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7dc004ce0 con 0x7fb7ec06c650 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.198+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3709381204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7ec19f010 con 0x7fb7ec06c650 2024-05-27T08:46:04.200 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.198+0000 7fb7e2ffd700 1 -- 172.21.15.77:0/3709381204 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc005550 con 0x7fb7ec06c650 2024-05-27T08:46:04.201 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.198+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3709381204 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7ec1a6890 con 0x7fb7ec06c650 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.200+0000 7fb7e2ffd700 1 -- 172.21.15.77:0/3709381204 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb7dc007ca0 con 0x7fb7ec06c650 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.200+0000 7fb7e2ffd700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb7d80b2a70 0x7fb7d80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.201+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb7d80b2a70 0x7fb7d80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.201+0000 7fb7e2ffd700 1 -- 172.21.15.77:0/3709381204 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fb7dc0e24d0 con 0x7fb7ec06c650 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.201+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb7d80b2a70 0x7fb7d80b4f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800b5c0 tx=0x7fb7e8005960 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.201+0000 7fb7fe716b80 1 Processor -- start 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7fe716b80 1 -- start start 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dec2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe000 0x55c28debe480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 0x55c28decaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.203 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c28de17e70 con 0x55c28dec2b30 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c28de07a10 con 0x55c28debe9c0 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c28de0ba50 con 0x55c28debe000 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dec2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dec2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60838/0 (socket says 172.21.15.77:60838) 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1460974408 learned_addr learned my addr 172.21.15.77:0/1460974408 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f141b700 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe000 0x55c28debe480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f241d700 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 0x55c28decaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1460974408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe000 msgr2=0x55c28debe480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe000 0x55c28debe480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1460974408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 msgr2=0x55c28decaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 0x55c28decaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1460974408 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28ddb10b0 con 0x55c28dec2b30 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f141b700 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe000 0x55c28debe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:46:04.204 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dec2f10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8015000 tx=0x7fb7e801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7141404b23afb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:46:04.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7d7fff700 1 -- 172.21.15.77:0/1460974408 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8035070 con 0x55c28dec2b30 2024-05-27T08:46:04.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.202+0000 7fb7d7fff700 1 -- 172.21.15.77:0/1460974408 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80259c0 con 0x55c28dec2b30 2024-05-27T08:46:04.205 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7d7fff700 1 -- 172.21.15.77:0/1460974408 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800fb00 con 0x55c28dec2b30 2024-05-27T08:46:04.208 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1460974408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 msgr2=0x55c28dec2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dec2f10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8015000 tx=0x7fb7e801f000 comp rx=0 tx=0).stop 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1460974408 shutdown_connections 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 0x55c28decaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dec2f10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1460974408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe000 0x55c28debe480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1460974408 >> 172.21.15.77:0/1460974408 conn(0x55c28dde5340 msgr2=0x55c28deb48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1460974408 shutdown_connections 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.203+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1460974408 wait complete. 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 1 -- start start 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 1 Processor -- start 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 0x55c28dec7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.209 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28dec2b30 0x55c28dec7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c28dd0e0a0 con 0x55c28debe000 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x55c28dd6fea0 con 0x55c28debe9c0 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c28decb050 con 0x55c28dec2b30 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7fe716b80 10 client.0 _ll_register_callbacks cb 0x55c28dda2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7f1c1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7f1c1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60840/0 (socket says 172.21.15.77:60840) 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.204+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1496932298 learned_addr learned my addr 172.21.15.77:0/1496932298 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f141b700 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 0x55c28dec7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f241d700 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28dec2b30 0x55c28dec7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1496932298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28dec2b30 msgr2=0x55c28dec7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28dec2b30 0x55c28dec7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1496932298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 msgr2=0x55c28dec7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:46:04.210 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 0x55c28dec7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1496932298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28dd579d0 con 0x55c28debe000 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f241d700 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28dec2b30 0x55c28dec7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deca6d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009350 tx=0x7fb7e800f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31955950c1e0fa93 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7d57fa700 1 -- 172.21.15.77:0/1496932298 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8040060 con 0x55c28debe000 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7d57fa700 1 -- 172.21.15.77:0/1496932298 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fb7e8025650 con 0x55c28debe000 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1496932298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e80259c0 con 0x55c28debe000 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7d57fa700 1 -- 172.21.15.77:0/1496932298 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb7e8035040 con 0x55c28debe000 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7fe716b80 10 client.15717 fetch_fsmap learned FSMap version 37 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1496932298 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c28ddc1860 con 0x55c28debe000 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.205+0000 7fb7d57fa700 1 -- 172.21.15.77:0/1496932298 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80259c0 con 0x55c28debe000 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.206+0000 7fb7d57fa700 1 -- 172.21.15.77:0/1496932298 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb7e80176c0 con 0x55c28debe000 2024-05-27T08:46:04.211 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.206+0000 7fb7fe716b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.206+0000 7fb7fe716b80 10 client.15717 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.206+0000 7fb7fe716b80 20 client.15717 populate_metadata read hostname 'smithi077' 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.206+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c28ded7a70 0x55c28ded9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.206+0000 7fb7fe716b80 4 client.15717 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.206+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1496932298 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c28de19dc0 con 0x55c28ded7a70 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.206+0000 7fb7f141b700 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c28ded7a70 0x55c28ded9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.207+0000 7fb7d57fa700 10 client.15717 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.207+0000 7fb7f141b700 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c28ded7a70 0x55c28ded9e60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541558b73a92e864 server_cookie=1e0696790ce80c13 in_seq=0 out_seq=0 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7d57fa700 1 -- 172.21.15.77:0/1496932298 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c28de19dc0 con 0x55c28ded7a70 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7d57fa700 10 client.15717 handle_command_reply: tid=0 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 client.15717 shutdown 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1496932298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 msgr2=0x55c28deca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.212 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deca6d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009350 tx=0x7fb7e800f500 comp rx=0 tx=0).stop 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1496932298 shutdown_connections 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c28ded7a70 0x55c28ded9e60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28dec2b30 0x55c28dec7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deca6d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7d57fa700 0 client.15717 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1496932298 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe9c0 0x55c28dec7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1496932298 >> 172.21.15.77:0/1496932298 conn(0x55c28dde5340 msgr2=0x55c28deb8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1496932298 shutdown_connections 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1496932298 wait complete. 2024-05-27T08:46:04.213 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.208+0000 7fb7fe716b80 20 client.15717 trim_cache size 0 max 16384 2024-05-27T08:46:04.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.228+0000 7fb7fe716b80 1 Processor -- start 2024-05-27T08:46:04.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.228+0000 7fb7fe716b80 1 -- start start 2024-05-27T08:46:04.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deaee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 0x55c28dec3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28dec2b30 0x55c28dec4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1abd30 con 0x55c28debe000 2024-05-27T08:46:04.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1ad930 con 0x55c28dec2b30 2024-05-27T08:46:04.230 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1ac400 con 0x55c28debe9c0 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deaee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deaee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60846/0 (socket says 172.21.15.77:60846) 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1194891863 learned_addr learned my addr 172.21.15.77:0/1194891863 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f141b700 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 0x55c28dec3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1194891863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 msgr2=0x55c28dec3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 0x55c28dec3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1194891863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28dec2b30 msgr2=0x55c28dec4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f241d700 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28dec2b30 0x55c28dec4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28dec2b30 0x55c28dec4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 -- 172.21.15.77:0/1194891863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28debf040 con 0x55c28debe000 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f241d700 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28dec2b30 0x55c28dec4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deaee70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e400 tx=0x7fb7e8025870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c584ef539ee843bb server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:46:04.231 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7d77fe700 1 -- 172.21.15.77:0/1194891863 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8040040 con 0x55c28debe000 2024-05-27T08:46:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7d77fe700 1 -- 172.21.15.77:0/1194891863 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80178a0 con 0x55c28debe000 2024-05-27T08:46:04.232 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.229+0000 7fb7d77fe700 1 -- 172.21.15.77:0/1194891863 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800e520 con 0x55c28debe000 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1194891863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 msgr2=0x55c28deaee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deaee70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e400 tx=0x7fb7e8025870 comp rx=0 tx=0).stop 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1194891863 shutdown_connections 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28dec2b30 0x55c28dec4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28debe000 0x55c28deaee70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/1194891863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 0x55c28dec3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1194891863 >> 172.21.15.77:0/1194891863 conn(0x55c28dde5340 msgr2=0x55c28dec37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1194891863 shutdown_connections 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.230+0000 7fb7fe716b80 1 -- 172.21.15.77:0/1194891863 wait complete. 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 1 -- start start 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 1 Processor -- start 2024-05-27T08:46:04.235 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe000 0x55c28dedbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 0x55c28dedc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dedc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1ad1a0 con 0x55c28dec2b30 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1acad0 con 0x55c28debe000 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec1a9cf0 con 0x55c28debe9c0 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7f241d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dedc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7fe716b80 10 client.0 _ll_register_callbacks cb 0x55c28dda2a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7f241d700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dedc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:60850/0 (socket says 172.21.15.77:60850) 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7f241d700 1 -- 172.21.15.77:0/4001884182 learned_addr learned my addr 172.21.15.77:0/4001884182 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7f141b700 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 0x55c28dedc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.231+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe000 0x55c28dedbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7f241d700 1 -- 172.21.15.77:0/4001884182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 msgr2=0x55c28dedc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7f241d700 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 0x55c28dedc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.236 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7f241d700 1 -- 172.21.15.77:0/4001884182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe000 msgr2=0x55c28dedbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7f241d700 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe000 0x55c28dedbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7f241d700 1 -- 172.21.15.77:0/4001884182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28dec3280 con 0x55c28dec2b30 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7f241d700 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dedc7e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400aa20 tx=0x7fb7e400ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a83d4aa601eea server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7d57fa700 1 -- 172.21.15.77:0/4001884182 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4015760 con 0x55c28dec2b30 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7d57fa700 1 -- 172.21.15.77:0/4001884182 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8586+0+0 (secure 0 0 0) 0x7fb7e4015900 con 0x55c28dec2b30 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 1 -- 172.21.15.77:0/4001884182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e80178a0 con 0x55c28dec2b30 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7d57fa700 1 -- 172.21.15.77:0/4001884182 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb7e4024050 con 0x55c28dec2b30 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 10 client.15723 fetch_fsmap learned FSMap version 37 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 1 -- 172.21.15.77:0/4001884182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c28df43600 con 0x55c28dec2b30 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7d57fa700 1 -- 172.21.15.77:0/4001884182 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80178a0 con 0x55c28dec2b30 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7d57fa700 1 -- 172.21.15.77:0/4001884182 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7fb7e4016c90 con 0x55c28dec2b30 2024-05-27T08:46:04.237 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 10 client.15723 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:46:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 10 client.15723 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:46:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 20 client.15723 populate_metadata read hostname 'smithi077' 2024-05-27T08:46:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c28df43e50 0x55c28df46270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:46:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 4 client.15723 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cc9d5d70", "release": true, "await": true}] 2024-05-27T08:46:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.232+0000 7fb7fe716b80 1 -- 172.21.15.77:0/4001884182 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cc9d5d70", "release": true, "await": true}) -- 0x55c28de135a0 con 0x55c28df43e50 2024-05-27T08:46:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.233+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c28df43e50 0x55c28df46270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:46:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.233+0000 7fb7f1c1c700 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c28df43e50 0x55c28df46270 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c00fa4e985f2637 server_cookie=c8958c3601fd87d9 in_seq=0 out_seq=0 2024-05-27T08:46:04.238 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.233+0000 7fb7d57fa700 10 client.15723 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:46:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7d57fa700 1 -- 172.21.15.77:0/4001884182 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c28de135a0 con 0x55c28df43e50 2024-05-27T08:46:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7d57fa700 10 client.15723 handle_command_reply: tid=0 2024-05-27T08:46:04.239 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 client.15723 shutdown 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 -- 172.21.15.77:0/4001884182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 msgr2=0x55c28dedc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dedc7e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400aa20 tx=0x7fb7e400ab00 comp rx=0 tx=0).stop 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 -- 172.21.15.77:0/4001884182 shutdown_connections 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x55c28df43e50 0x55c28df46270 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c28dec2b30 0x55c28dedc7e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55c28debe000 0x55c28dedbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 --2- 172.21.15.77:0/4001884182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c28debe9c0 0x55c28dedc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7d57fa700 0 client.15723 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 -- 172.21.15.77:0/4001884182 >> 172.21.15.77:0/4001884182 conn(0x55c28dde5340 msgr2=0x55c28dec5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 -- 172.21.15.77:0/4001884182 shutdown_connections 2024-05-27T08:46:04.240 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 1 -- 172.21.15.77:0/4001884182 wait complete. 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.239+0000 7fb7fe716b80 20 client.15723 trim_cache size 0 max 16384 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 37, 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 24, 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout: "cc9d5d70": { 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 24, 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:46:04.241 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 149.8, 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:46:04.242 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:46:04.243 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:46:04.243 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:46:04.243 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:46:04.243 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:46:04.243 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb7d80b2a70 msgr2=0x7fb7d80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb7d80b2a70 0x7fb7d80b4f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800b5c0 tx=0x7fb7e8005960 comp rx=0 tx=0).stop 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec06c650 msgr2=0x7fb7ec19a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec06c650 0x7fb7ec19a250 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00c920 tx=0x7fb7dc00cc30 comp rx=0 tx=0).stop 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3709381204 shutdown_connections 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb7d80b2a70 0x7fb7d80b4f30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7ec1095b0 0x7fb7ec19e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb7ec06bc90 0x7fb7ec199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 --2- 172.21.15.77:0/3709381204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7ec06c650 0x7fb7ec19a250 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3709381204 >> 172.21.15.77:0/3709381204 conn(0x7fb7ec079410 msgr2=0x7fb7ec1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3709381204 shutdown_connections 2024-05-27T08:46:04.244 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:46:04.240+0000 7fb7f3e80700 1 -- 172.21.15.77:0/3709381204 wait complete. 2024-05-27T08:46:04.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 264967 3.97 MB/sec execute 238 sec latency 29794.547 ms 2024-05-27T08:46:04.961 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cc9d5d70', total seconds elapsed: 35.4 2024-05-27T08:46:04.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:04 smithi092 ceph-mon[36049]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T08:46:04.991 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:04 smithi077 ceph-mon[32007]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T08:46:05.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:04 smithi179 ceph-mon[36481]: osdmap e89: 12 total, 12 up, 12 in 2024-05-27T08:46:05.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 266510 4.00 MB/sec execute 239 sec latency 10.437 ms 2024-05-27T08:46:05.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:05 smithi092 ceph-mon[36049]: pgmap v240: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 780 KiB/s wr, 69 op/s 2024-05-27T08:46:06.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:05 smithi179 ceph-mon[36481]: pgmap v240: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 780 KiB/s wr, 69 op/s 2024-05-27T08:46:06.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:05 smithi077 ceph-mon[32007]: pgmap v240: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 780 KiB/s wr, 69 op/s 2024-05-27T08:46:06.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 267752 4.00 MB/sec execute 240 sec latency 159.801 ms 2024-05-27T08:46:07.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 267752 3.99 MB/sec execute 241 sec latency 1159.896 ms 2024-05-27T08:46:08.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:07 smithi179 ceph-mon[36481]: pgmap v241: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 780 KiB/s wr, 69 op/s 2024-05-27T08:46:08.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:07 smithi077 ceph-mon[32007]: pgmap v241: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 780 KiB/s wr, 69 op/s 2024-05-27T08:46:08.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:07 smithi092 ceph-mon[36049]: pgmap v241: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 780 KiB/s wr, 69 op/s 2024-05-27T08:46:08.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 267752 3.97 MB/sec execute 242 sec latency 2159.970 ms 2024-05-27T08:46:09.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 267752 3.95 MB/sec execute 243 sec latency 3160.038 ms 2024-05-27T08:46:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:09 smithi179 ceph-mon[36481]: pgmap v242: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 KiB/s wr, 117 op/s 2024-05-27T08:46:10.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:09 smithi077 ceph-mon[32007]: pgmap v242: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 KiB/s wr, 117 op/s 2024-05-27T08:46:10.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:09 smithi092 ceph-mon[36049]: pgmap v242: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 KiB/s wr, 117 op/s 2024-05-27T08:46:10.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 267752 3.94 MB/sec execute 244 sec latency 4160.085 ms 2024-05-27T08:46:11.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 269829 3.98 MB/sec execute 245 sec latency 4232.398 ms 2024-05-27T08:46:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:11 smithi179 ceph-mon[36481]: pgmap v243: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 KiB/s wr, 117 op/s 2024-05-27T08:46:11.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:11 smithi179 ceph-mon[36481]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T08:46:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:11 smithi077 ceph-mon[32007]: pgmap v243: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 KiB/s wr, 117 op/s 2024-05-27T08:46:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:11 smithi077 ceph-mon[32007]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T08:46:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:11 smithi092 ceph-mon[36049]: pgmap v243: 97 pgs: 97 active+clean; 181 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 808 KiB/s wr, 117 op/s 2024-05-27T08:46:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:11 smithi092 ceph-mon[36049]: osdmap e90: 12 total, 12 up, 12 in 2024-05-27T08:46:12.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 271055 3.98 MB/sec execute 246 sec latency 15.555 ms 2024-05-27T08:46:13.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 271294 3.97 MB/sec execute 247 sec latency 754.753 ms 2024-05-27T08:46:13.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:13 smithi077 ceph-mon[32007]: pgmap v245: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-27T08:46:13.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:13 smithi092 ceph-mon[36049]: pgmap v245: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-27T08:46:14.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:13 smithi179 ceph-mon[36481]: pgmap v245: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-05-27T08:46:14.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 271294 3.95 MB/sec execute 248 sec latency 1754.790 ms 2024-05-27T08:46:15.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 271294 3.94 MB/sec execute 249 sec latency 2754.829 ms 2024-05-27T08:46:15.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:15 smithi077 ceph-mon[32007]: pgmap v246: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 KiB/s wr, 122 op/s 2024-05-27T08:46:15.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:15 smithi092 ceph-mon[36049]: pgmap v246: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 KiB/s wr, 122 op/s 2024-05-27T08:46:16.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:15 smithi179 ceph-mon[36481]: pgmap v246: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 KiB/s wr, 122 op/s 2024-05-27T08:46:16.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 273130 3.97 MB/sec execute 250 sec latency 2949.702 ms 2024-05-27T08:46:17.609 INFO:tasks.workunit.client.0.smithi077.stdout: 1 274315 3.97 MB/sec execute 251 sec latency 16.550 ms 2024-05-27T08:46:17.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:17 smithi077 ceph-mon[32007]: pgmap v247: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 KiB/s wr, 122 op/s 2024-05-27T08:46:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:17 smithi092 ceph-mon[36049]: pgmap v247: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 KiB/s wr, 122 op/s 2024-05-27T08:46:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:17 smithi179 ceph-mon[36481]: pgmap v247: 97 pgs: 97 active+clean; 182 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 947 KiB/s wr, 122 op/s 2024-05-27T08:46:18.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 274843 3.96 MB/sec execute 252 sec latency 578.228 ms 2024-05-27T08:46:19.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 274843 3.95 MB/sec execute 253 sec latency 1578.330 ms 2024-05-27T08:46:19.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:19 smithi077 ceph-mon[32007]: pgmap v248: 97 pgs: 97 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-27T08:46:19.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:19 smithi092 ceph-mon[36049]: pgmap v248: 97 pgs: 97 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-27T08:46:20.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:19 smithi179 ceph-mon[36481]: pgmap v248: 97 pgs: 97 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-27T08:46:20.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 274843 3.93 MB/sec execute 254 sec latency 2578.478 ms 2024-05-27T08:46:21.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 276722 3.97 MB/sec execute 255 sec latency 2767.655 ms 2024-05-27T08:46:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:21 smithi077 ceph-mon[32007]: pgmap v249: 97 pgs: 97 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-27T08:46:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:21 smithi092 ceph-mon[36049]: pgmap v249: 97 pgs: 97 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-27T08:46:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:21 smithi179 ceph-mon[36481]: pgmap v249: 97 pgs: 97 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 933 KiB/s rd, 737 KiB/s wr, 101 op/s 2024-05-27T08:46:22.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 277951 3.97 MB/sec execute 256 sec latency 18.628 ms 2024-05-27T08:46:23.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 278385 3.96 MB/sec execute 257 sec latency 629.375 ms 2024-05-27T08:46:23.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:23 smithi077 ceph-mon[32007]: pgmap v250: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 992 KiB/s wr, 139 op/s 2024-05-27T08:46:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:23 smithi092 ceph-mon[36049]: pgmap v250: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 992 KiB/s wr, 139 op/s 2024-05-27T08:46:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:23 smithi179 ceph-mon[36481]: pgmap v250: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 992 KiB/s wr, 139 op/s 2024-05-27T08:46:24.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 278385 3.95 MB/sec execute 258 sec latency 1629.449 ms 2024-05-27T08:46:25.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 278385 3.93 MB/sec execute 259 sec latency 2629.495 ms 2024-05-27T08:46:25.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:25 smithi077 ceph-mon[32007]: pgmap v251: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 700 KiB/s wr, 96 op/s 2024-05-27T08:46:25.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:25 smithi092 ceph-mon[36049]: pgmap v251: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 700 KiB/s wr, 96 op/s 2024-05-27T08:46:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:25 smithi179 ceph-mon[36481]: pgmap v251: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 700 KiB/s wr, 96 op/s 2024-05-27T08:46:26.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 280458 3.97 MB/sec execute 260 sec latency 2699.108 ms 2024-05-27T08:46:27.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 281670 3.97 MB/sec execute 261 sec latency 15.970 ms 2024-05-27T08:46:27.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:27 smithi077 ceph-mon[32007]: pgmap v252: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 700 KiB/s wr, 96 op/s 2024-05-27T08:46:27.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:27 smithi092 ceph-mon[36049]: pgmap v252: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 700 KiB/s wr, 96 op/s 2024-05-27T08:46:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:27 smithi179 ceph-mon[36481]: pgmap v252: 97 pgs: 97 active+clean; 189 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 700 KiB/s wr, 96 op/s 2024-05-27T08:46:28.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 281934 3.96 MB/sec execute 262 sec latency 722.703 ms 2024-05-27T08:46:28.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:28 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:46:29.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:28 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:46:29.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:28 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:46:29.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 281934 3.94 MB/sec execute 263 sec latency 1722.766 ms 2024-05-27T08:46:29.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:29 smithi092 ceph-mon[36049]: pgmap v253: 97 pgs: 97 active+clean; 192 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-05-27T08:46:29.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:29 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:46:29.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:29 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:46:29.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:29 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:46:30.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:29 smithi179 ceph-mon[36481]: pgmap v253: 97 pgs: 97 active+clean; 192 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-05-27T08:46:30.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:29 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:46:30.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:29 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:46:30.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:29 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:46:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:29 smithi077 ceph-mon[32007]: pgmap v253: 97 pgs: 97 active+clean; 192 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.0 MiB/s wr, 150 op/s 2024-05-27T08:46:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:29 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:46:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:29 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:46:30.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:29 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:46:30.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 281934 3.93 MB/sec execute 264 sec latency 2722.808 ms 2024-05-27T08:46:31.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 283857 3.96 MB/sec execute 265 sec latency 2911.543 ms 2024-05-27T08:46:31.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:31 smithi077 ceph-mon[32007]: pgmap v254: 97 pgs: 97 active+clean; 192 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 709 KiB/s wr, 105 op/s 2024-05-27T08:46:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:31 smithi092 ceph-mon[36049]: pgmap v254: 97 pgs: 97 active+clean; 192 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 709 KiB/s wr, 105 op/s 2024-05-27T08:46:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:31 smithi179 ceph-mon[36481]: pgmap v254: 97 pgs: 97 active+clean; 192 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 709 KiB/s wr, 105 op/s 2024-05-27T08:46:32.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 285005 3.96 MB/sec execute 266 sec latency 17.855 ms 2024-05-27T08:46:33.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 285476 3.96 MB/sec execute 267 sec latency 611.586 ms 2024-05-27T08:46:33.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:33 smithi092 ceph-mon[36049]: pgmap v255: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 996 KiB/s wr, 150 op/s 2024-05-27T08:46:34.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:33 smithi179 ceph-mon[36481]: pgmap v255: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 996 KiB/s wr, 150 op/s 2024-05-27T08:46:34.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:33 smithi077 ceph-mon[32007]: pgmap v255: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 996 KiB/s wr, 150 op/s 2024-05-27T08:46:34.610 INFO:tasks.workunit.client.0.smithi077.stdout: 1 285476 3.94 MB/sec execute 268 sec latency 1611.681 ms 2024-05-27T08:46:35.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 285476 3.93 MB/sec execute 269 sec latency 2611.794 ms 2024-05-27T08:46:35.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:35 smithi092 ceph-mon[36049]: pgmap v256: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 640 KiB/s wr, 99 op/s 2024-05-27T08:46:36.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:35 smithi179 ceph-mon[36481]: pgmap v256: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 640 KiB/s wr, 99 op/s 2024-05-27T08:46:36.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:35 smithi077 ceph-mon[32007]: pgmap v256: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 640 KiB/s wr, 99 op/s 2024-05-27T08:46:36.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 287312 3.96 MB/sec execute 270 sec latency 2804.114 ms 2024-05-27T08:46:37.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 288520 3.96 MB/sec execute 271 sec latency 26.397 ms 2024-05-27T08:46:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:37 smithi092 ceph-mon[36049]: pgmap v257: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 640 KiB/s wr, 99 op/s 2024-05-27T08:46:38.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:37 smithi179 ceph-mon[36481]: pgmap v257: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 640 KiB/s wr, 99 op/s 2024-05-27T08:46:38.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:37 smithi077 ceph-mon[32007]: pgmap v257: 97 pgs: 97 active+clean; 195 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 640 KiB/s wr, 99 op/s 2024-05-27T08:46:38.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 289024 3.95 MB/sec execute 272 sec latency 605.523 ms 2024-05-27T08:46:39.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 289024 3.94 MB/sec execute 273 sec latency 1605.626 ms 2024-05-27T08:46:39.907 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:39 smithi179 ceph-mon[36481]: pgmap v258: 97 pgs: 97 active+clean; 201 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-27T08:46:39.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:39 smithi092 ceph-mon[36049]: pgmap v258: 97 pgs: 97 active+clean; 201 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-27T08:46:40.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:39 smithi077 ceph-mon[32007]: pgmap v258: 97 pgs: 97 active+clean; 201 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-05-27T08:46:40.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 289024 3.93 MB/sec execute 274 sec latency 2605.664 ms 2024-05-27T08:46:41.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 291087 3.96 MB/sec execute 275 sec latency 2673.141 ms 2024-05-27T08:46:41.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:41 smithi092 ceph-mon[36049]: pgmap v259: 97 pgs: 97 active+clean; 201 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 947 KiB/s wr, 100 op/s 2024-05-27T08:46:42.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:41 smithi179 ceph-mon[36481]: pgmap v259: 97 pgs: 97 active+clean; 201 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 947 KiB/s wr, 100 op/s 2024-05-27T08:46:42.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:41 smithi077 ceph-mon[32007]: pgmap v259: 97 pgs: 97 active+clean; 201 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 947 KiB/s wr, 100 op/s 2024-05-27T08:46:42.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 292381 3.97 MB/sec execute 276 sec latency 15.916 ms 2024-05-27T08:46:43.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 292567 3.95 MB/sec execute 277 sec latency 878.636 ms 2024-05-27T08:46:43.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:43 smithi092 ceph-mon[36049]: pgmap v260: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-05-27T08:46:44.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:43 smithi179 ceph-mon[36481]: pgmap v260: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-05-27T08:46:44.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:43 smithi077 ceph-mon[32007]: pgmap v260: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-05-27T08:46:44.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 292567 3.94 MB/sec execute 278 sec latency 1878.741 ms 2024-05-27T08:46:45.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 292567 3.92 MB/sec execute 279 sec latency 2878.814 ms 2024-05-27T08:46:45.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:45 smithi092 ceph-mon[36049]: pgmap v261: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-27T08:46:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:45 smithi179 ceph-mon[36481]: pgmap v261: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-27T08:46:46.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:45 smithi077 ceph-mon[32007]: pgmap v261: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-27T08:46:46.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 294427 3.95 MB/sec execute 280 sec latency 3067.096 ms 2024-05-27T08:46:47.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 295603 3.95 MB/sec execute 281 sec latency 17.301 ms 2024-05-27T08:46:47.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:47 smithi092 ceph-mon[36049]: pgmap v262: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-27T08:46:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:47 smithi179 ceph-mon[36481]: pgmap v262: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-27T08:46:48.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:47 smithi077 ceph-mon[32007]: pgmap v262: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-05-27T08:46:48.611 INFO:tasks.workunit.client.0.smithi077.stdout: 1 296117 3.95 MB/sec execute 282 sec latency 725.639 ms 2024-05-27T08:46:49.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 296117 3.94 MB/sec execute 283 sec latency 1725.734 ms 2024-05-27T08:46:49.931 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:49 smithi077 ceph-mon[32007]: pgmap v263: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-05-27T08:46:49.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:49 smithi092 ceph-mon[36049]: pgmap v263: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-05-27T08:46:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:49 smithi179 ceph-mon[36481]: pgmap v263: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-05-27T08:46:50.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 296117 3.92 MB/sec execute 284 sec latency 2725.783 ms 2024-05-27T08:46:51.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 298053 3.95 MB/sec execute 285 sec latency 2914.871 ms 2024-05-27T08:46:51.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:51 smithi092 ceph-mon[36049]: pgmap v264: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-27T08:46:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:51 smithi179 ceph-mon[36481]: pgmap v264: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-27T08:46:52.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:51 smithi077 ceph-mon[32007]: pgmap v264: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-05-27T08:46:52.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 299235 3.95 MB/sec execute 286 sec latency 16.328 ms 2024-05-27T08:46:53.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 299665 3.95 MB/sec execute 287 sec latency 781.690 ms 2024-05-27T08:46:53.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:53 smithi092 ceph-mon[36049]: pgmap v265: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-05-27T08:46:54.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:53 smithi179 ceph-mon[36481]: pgmap v265: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-05-27T08:46:54.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:53 smithi077 ceph-mon[32007]: pgmap v265: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-05-27T08:46:54.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 299665 3.93 MB/sec execute 288 sec latency 1781.789 ms 2024-05-27T08:46:55.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 299665 3.92 MB/sec execute 289 sec latency 2781.848 ms 2024-05-27T08:46:55.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:55 smithi092 ceph-mon[36049]: pgmap v266: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 108 op/s 2024-05-27T08:46:56.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:55 smithi179 ceph-mon[36481]: pgmap v266: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 108 op/s 2024-05-27T08:46:56.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:55 smithi077 ceph-mon[32007]: pgmap v266: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 108 op/s 2024-05-27T08:46:56.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 301755 3.95 MB/sec execute 290 sec latency 2850.224 ms 2024-05-27T08:46:57.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 303028 3.96 MB/sec execute 291 sec latency 20.973 ms 2024-05-27T08:46:57.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:57 smithi092 ceph-mon[36049]: pgmap v267: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 108 op/s 2024-05-27T08:46:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:57 smithi179 ceph-mon[36481]: pgmap v267: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 108 op/s 2024-05-27T08:46:58.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:57 smithi077 ceph-mon[32007]: pgmap v267: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 893 KiB/s wr, 108 op/s 2024-05-27T08:46:58.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 303214 3.95 MB/sec execute 292 sec latency 880.028 ms 2024-05-27T08:46:59.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 303214 3.93 MB/sec execute 293 sec latency 1880.149 ms 2024-05-27T08:46:59.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:46:59 smithi092 ceph-mon[36049]: pgmap v268: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-27T08:47:00.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:46:59 smithi179 ceph-mon[36481]: pgmap v268: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-27T08:47:00.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:46:59 smithi077 ceph-mon[32007]: pgmap v268: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-05-27T08:47:00.305 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.3 2024-05-27T08:47:00.306 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:47:00.559 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:47:00.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 303214 3.92 MB/sec execute 294 sec latency 2880.192 ms 2024-05-27T08:47:01.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 304296 3.93 MB/sec execute 295 sec latency 3068.709 ms 2024-05-27T08:47:01.988 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:01 smithi092 ceph-mon[36049]: pgmap v269: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 972 KiB/s wr, 122 op/s 2024-05-27T08:47:02.034 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:01 smithi077 ceph-mon[32007]: pgmap v269: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 972 KiB/s wr, 122 op/s 2024-05-27T08:47:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:01 smithi179 ceph-mon[36481]: pgmap v269: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 972 KiB/s wr, 122 op/s 2024-05-27T08:47:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.435+0000 7f94e3fb3700 1 -- 172.21.15.77:0/672383378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc101b30 msgr2=0x7f94dc101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.435+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/672383378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc101b30 0x7f94dc101fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f94cc009990 tx=0x7f94cc009ca0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.435+0000 7f94e3fb3700 1 -- 172.21.15.77:0/672383378 shutdown_connections 2024-05-27T08:47:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.435+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/672383378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1025d0 0x7f94dc1122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.435+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/672383378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94dc104450 0x7f94dc104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.435+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/672383378 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc101b30 0x7f94dc101fb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.435+0000 7f94e3fb3700 1 -- 172.21.15.77:0/672383378 >> 172.21.15.77:0/672383378 conn(0x7f94dc078ab0 msgr2=0x7f94dc0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.436+0000 7f94e3fb3700 1 -- 172.21.15.77:0/672383378 shutdown_connections 2024-05-27T08:47:02.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.436+0000 7f94e3fb3700 1 -- 172.21.15.77:0/672383378 wait complete. 2024-05-27T08:47:02.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.436+0000 7f94e3fb3700 1 Processor -- start 2024-05-27T08:47:02.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.436+0000 7f94e3fb3700 1 -- start start 2024-05-27T08:47:02.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e3fb3700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94dc101b30 0x7f94dc19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:02.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e3fb3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1025d0 0x7f94dc19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:02.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e3fb3700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc104450 0x7f94dc1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:02.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e3fb3700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc115910 con 0x7f94dc101b30 2024-05-27T08:47:02.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e3fb3700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc115790 con 0x7f94dc104450 2024-05-27T08:47:02.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e3fb3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc115a90 con 0x7f94dc1025d0 2024-05-27T08:47:02.441 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e1d4f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94dc101b30 0x7f94dc19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:02.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e1d4f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94dc101b30 0x7f94dc19bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59416/0 (socket says 172.21.15.77:59416) 2024-05-27T08:47:02.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e2550700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc104450 0x7f94dc1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:02.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e1d4f700 1 -- 172.21.15.77:0/320846156 learned_addr learned my addr 172.21.15.77:0/320846156 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:02.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e154e700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1025d0 0x7f94dc19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:02.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e2550700 1 -- 172.21.15.77:0/320846156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1025d0 msgr2=0x7f94dc19c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:02.442 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e2550700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1025d0 0x7f94dc19c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e2550700 1 -- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94dc101b30 msgr2=0x7f94dc19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:02.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e2550700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94dc101b30 0x7f94dc19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e2550700 1 -- 172.21.15.77:0/320846156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc1a0df0 con 0x7f94dc104450 2024-05-27T08:47:02.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94e2550700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc104450 0x7f94dc1a06b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94d400efc0 tx=0x7f94d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:02.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94d2ffd700 1 -- 172.21.15.77:0/320846156 <== mon.2 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40092e0 con 0x7f94dc104450 2024-05-27T08:47:02.443 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.437+0000 7f94d2ffd700 1 -- 172.21.15.77:0/320846156 <== mon.2 v2:172.21.15.92:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94d4004bb0 con 0x7f94dc104450 2024-05-27T08:47:02.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.438+0000 7f94d2ffd700 1 -- 172.21.15.77:0/320846156 <== mon.2 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4004020 con 0x7f94dc104450 2024-05-27T08:47:02.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.438+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc1a1080 con 0x7f94dc104450 2024-05-27T08:47:02.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.438+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc1a8960 con 0x7f94dc104450 2024-05-27T08:47:02.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.439+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc102b40 con 0x7f94dc104450 2024-05-27T08:47:02.444 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.440+0000 7f94d2ffd700 1 -- 172.21.15.77:0/320846156 <== mon.2 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f94d401b030 con 0x7f94dc104450 2024-05-27T08:47:02.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.441+0000 7f94d2ffd700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94c80b29f0 0x7f94c80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:02.445 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.441+0000 7f94e1d4f700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94c80b29f0 0x7f94c80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:02.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.441+0000 7f94d2ffd700 1 -- 172.21.15.77:0/320846156 <== mon.2 v2:172.21.15.92:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8542+0+0 (secure 0 0 0) 0x7f94d40e2110 con 0x7f94dc104450 2024-05-27T08:47:02.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.441+0000 7f94e1d4f700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94c80b29f0 0x7f94c80b4eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009680 tx=0x7f94d8009260 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:02.446 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.445+0000 7f94d2ffd700 1 -- 172.21.15.77:0/320846156 <== mon.2 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f94d4022020 con 0x7f94dc104450 2024-05-27T08:47:02.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 305751 3.94 MB/sec execute 296 sec latency 23.409 ms 2024-05-27T08:47:02.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.772+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94dc02e560 con 0x7f94dc104450 2024-05-27T08:47:02.773 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.773+0000 7f94d2ffd700 1 -- 172.21.15.77:0/320846156 <== mon.2 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11934 (secure 0 0 0) 0x7f94d4010040 con 0x7f94dc104450 2024-05-27T08:47:02.773 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:47:02.774 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:44:43.671254+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:47:02.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94c80b29f0 msgr2=0x7f94c80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:02.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94c80b29f0 0x7f94c80b4eb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009680 tx=0x7f94d8009260 comp rx=0 tx=0).stop 2024-05-27T08:47:02.775 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc104450 msgr2=0x7f94dc1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc104450 0x7f94dc1a06b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94d400efc0 tx=0x7f94d400c3a0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 shutdown_connections 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94c80b29f0 0x7f94c80b4eb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94dc104450 0x7f94dc1a06b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94dc101b30 0x7f94dc19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 --2- 172.21.15.77:0/320846156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94dc1025d0 0x7f94dc19c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 >> 172.21.15.77:0/320846156 conn(0x7f94dc078ab0 msgr2=0x7f94dc0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 shutdown_connections 2024-05-27T08:47:02.776 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:02.775+0000 7f94e3fb3700 1 -- 172.21.15.77:0/320846156 wait complete. 2024-05-27T08:47:02.777 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 37 2024-05-27T08:47:03.466 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 90 --expiration 144.3 --await-for 45.0' 2024-05-27T08:47:03.466 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db / --timeout 90 --expiration 144.3 --await-for 45.0 2024-05-27T08:47:03.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306757 3.94 MB/sec execute 297 sec latency 420.049 ms 2024-05-27T08:47:03.729 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:47:03.976 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:03 smithi077 ceph-mon[32007]: pgmap v270: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-05-27T08:47:03.977 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:03 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/320846156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:47:03.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:03 smithi092 ceph-mon[36049]: pgmap v270: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-05-27T08:47:03.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:03 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/320846156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:47:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:03 smithi179 ceph-mon[36481]: pgmap v270: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-05-27T08:47:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:03 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/320846156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:47:04.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306757 3.93 MB/sec execute 298 sec latency 1420.088 ms 2024-05-27T08:47:04.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:04 smithi092 ceph-mon[36049]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T08:47:05.069 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:04 smithi077 ceph-mon[32007]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T08:47:05.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:04 smithi179 ceph-mon[36481]: osdmap e91: 12 total, 12 up, 12 in 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 -- 172.21.15.77:0/1244734225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848104200 msgr2=0x7f7848108160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 --2- 172.21.15.77:0/1244734225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848104200 0x7f7848108160 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f78300099c0 tx=0x7f7830009cd0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 -- 172.21.15.77:0/1244734225 shutdown_connections 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 --2- 172.21.15.77:0/1244734225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7848108840 0x7f7848110450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 --2- 172.21.15.77:0/1244734225 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7848103850 0x7f7848103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 --2- 172.21.15.77:0/1244734225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848104200 0x7f7848108160 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 -- 172.21.15.77:0/1244734225 >> 172.21.15.77:0/1244734225 conn(0x7f784807a610 msgr2=0x7f784807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 -- 172.21.15.77:0/1244734225 shutdown_connections 2024-05-27T08:47:05.423 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.420+0000 7f784d299700 1 -- 172.21.15.77:0/1244734225 wait complete. 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f784d299700 1 Processor -- start 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f784d299700 1 -- start start 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f784d299700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7848103850 0x7f784819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f784d299700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7848104200 0x7f784819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f784d299700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848108840 0x7f7848199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f784d299700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7848113790 con 0x7f7848108840 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f784d299700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7848113610 con 0x7f7848104200 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f784d299700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7848113910 con 0x7f7848103850 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f78477fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848108840 0x7f7848199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f78477fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848108840 0x7f7848199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59436/0 (socket says 172.21.15.77:59436) 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f78477fe700 1 -- 172.21.15.77:0/1705596270 learned_addr learned my addr 172.21.15.77:0/1705596270 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:05.424 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f783e7fc700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7848104200 0x7f784819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.421+0000 7f7846ffd700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7848103850 0x7f784819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f78477fe700 1 -- 172.21.15.77:0/1705596270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7848103850 msgr2=0x7f784819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f78477fe700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7848103850 0x7f784819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f78477fe700 1 -- 172.21.15.77:0/1705596270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7848104200 msgr2=0x7f784819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f78477fe700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7848104200 0x7f784819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f78477fe700 1 -- 172.21.15.77:0/1705596270 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784819a1b0 con 0x7f7848108840 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f78477fe700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848108840 0x7f7848199a50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7840007ee0 tx=0x7f784000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f7844ff9700 1 -- 172.21.15.77:0/1705596270 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840004bb0 con 0x7f7848108840 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f7844ff9700 1 -- 172.21.15.77:0/1705596270 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840004d50 con 0x7f7848108840 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f7844ff9700 1 -- 172.21.15.77:0/1705596270 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840005920 con 0x7f7848108840 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f784d299700 1 -- 172.21.15.77:0/1705596270 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784819a4a0 con 0x7f7848108840 2024-05-27T08:47:05.425 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.422+0000 7f784d299700 1 -- 172.21.15.77:0/1705596270 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a6710 con 0x7f7848108840 2024-05-27T08:47:05.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.424+0000 7f7844ff9700 1 -- 172.21.15.77:0/1705596270 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f784001a020 con 0x7f7848108840 2024-05-27T08:47:05.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.425+0000 7f7844ff9700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f78340b2b20 0x7f78340b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.425+0000 7f7846ffd700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f78340b2b20 0x7f78340b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.428 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.425+0000 7f7844ff9700 1 -- 172.21.15.77:0/1705596270 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f78400e1c30 con 0x7f7848108840 2024-05-27T08:47:05.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7857b2fb80 1 Processor -- start 2024-05-27T08:47:05.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7846ffd700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f78340b2b20 0x7f78340b4fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7848107c50 tx=0x7f7838009190 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:05.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7857b2fb80 1 -- start start 2024-05-27T08:47:05.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00569720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00563df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00570170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564a004c2e20 con 0x564a00563970 2024-05-27T08:47:05.429 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564a004b29c0 con 0x564a00564330 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564a004b6a00 con 0x564a00569340 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f783e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00563df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f783e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00563df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59440/0 (socket says 172.21.15.77:59440) 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f783e7fc700 1 -- 172.21.15.77:0/2970311755 learned_addr learned my addr 172.21.15.77:0/2970311755 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f7846ffd700 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00569720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f78477fe700 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00570170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f783e7fc700 1 -- 172.21.15.77:0/2970311755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 msgr2=0x564a00569720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f783e7fc700 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00569720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f783e7fc700 1 -- 172.21.15.77:0/2970311755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 msgr2=0x564a00570170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f783e7fc700 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00570170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.430 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.426+0000 7f783e7fc700 1 -- 172.21.15.77:0/2970311755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a00402a60 con 0x564a00563970 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.427+0000 7f78477fe700 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00570170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.427+0000 7f7846ffd700 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00569720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.427+0000 7f783e7fc700 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00563df0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f784806cd30 tx=0x7f7830009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bbc231a4a938c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.427+0000 7f783d7fa700 1 -- 172.21.15.77:0/2970311755 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783000bac0 con 0x564a00563970 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.427+0000 7f783d7fa700 1 -- 172.21.15.77:0/2970311755 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7830026070 con 0x564a00563970 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.427+0000 7f783d7fa700 1 -- 172.21.15.77:0/2970311755 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830005270 con 0x564a00563970 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- 172.21.15.77:0/2970311755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 msgr2=0x564a00563df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00563df0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f784806cd30 tx=0x7f7830009670 comp rx=0 tx=0).stop 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- 172.21.15.77:0/2970311755 shutdown_connections 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00570170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00569720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.431 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/2970311755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00563df0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- 172.21.15.77:0/2970311755 >> 172.21.15.77:0/2970311755 conn(0x564a00491210 msgr2=0x564a0055e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:05.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- 172.21.15.77:0/2970311755 shutdown_connections 2024-05-27T08:47:05.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- 172.21.15.77:0/2970311755 wait complete. 2024-05-27T08:47:05.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- start start 2024-05-27T08:47:05.432 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 Processor -- start 2024-05-27T08:47:05.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00563970 0x564a0056fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.435 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00564330 0x564a0056d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a0056d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.436 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564a003b9130 con 0x564a00564330 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564a00454950 con 0x564a00563970 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564a00570710 con 0x564a00569340 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.428+0000 7f7857b2fb80 10 client.0 _ll_register_callbacks cb 0x564a0044dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00564330 0x564a0056d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f7846ffd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00563970 0x564a0056fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00564330 0x564a0056d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59454/0 (socket says 172.21.15.77:59454) 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 -- 172.21.15.77:0/215642530 learned_addr learned my addr 172.21.15.77:0/215642530 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f78477fe700 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a0056d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 -- 172.21.15.77:0/215642530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 msgr2=0x564a0056d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a0056d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.437 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 -- 172.21.15.77:0/215642530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00563970 msgr2=0x564a0056fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00563970 0x564a0056fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 -- 172.21.15.77:0/215642530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a00451820 con 0x564a00564330 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f78477fe700 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a0056d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f7846ffd700 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00563970 0x564a0056fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f783e7fc700 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00564330 0x564a0056d080 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7830006010 tx=0x7f7830003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79336554d5b78e23 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f782e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f782e7fc700 1 -- 172.21.15.77:0/215642530 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830005070 con 0x564a00564330 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f782e7fc700 1 -- 172.21.15.77:0/215642530 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f7830005210 con 0x564a00564330 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f7857b2fb80 1 -- 172.21.15.77:0/215642530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7830026070 con 0x564a00564330 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f782e7fc700 1 -- 172.21.15.77:0/215642530 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7830005410 con 0x564a00564330 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f7857b2fb80 10 client.15741 fetch_fsmap learned FSMap version 37 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.429+0000 7f7857b2fb80 1 -- 172.21.15.77:0/215642530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564a0057ad00 con 0x564a00564330 2024-05-27T08:47:05.438 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f782e7fc700 1 -- 172.21.15.77:0/215642530 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7830026070 con 0x564a00564330 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f782e7fc700 1 -- 172.21.15.77:0/215642530 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f7830032400 con 0x564a00564330 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f7857b2fb80 10 client.15741 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f7857b2fb80 10 client.15741 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f7857b2fb80 20 client.15741 populate_metadata read hostname 'smithi077' 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564a0057d1d0 0x564a0057f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f7857b2fb80 4 client.15741 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f7857b2fb80 1 -- 172.21.15.77:0/215642530 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a0052fb70 con 0x564a0057d1d0 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f7846ffd700 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564a0057d1d0 0x564a0057f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f7846ffd700 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564a0057d1d0 0x564a0057f5c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce3b2219df9a596 server_cookie=3157be4629bb8825 in_seq=0 out_seq=0 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.430+0000 7f782e7fc700 10 client.15741 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f782e7fc700 1 -- 172.21.15.77:0/215642530 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a0052fb70 con 0x564a0057d1d0 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f782e7fc700 10 client.15741 handle_command_reply: tid=0 2024-05-27T08:47:05.439 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 client.15741 shutdown 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 -- 172.21.15.77:0/215642530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00564330 msgr2=0x564a0056d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00564330 0x564a0056d080 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7830006010 tx=0x7f7830003f40 comp rx=0 tx=0).stop 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 -- 172.21.15.77:0/215642530 shutdown_connections 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564a0057d1d0 0x564a0057f5c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a0056d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00563970 0x564a0056fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f782e7fc700 0 client.15741 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/215642530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00564330 0x564a0056d080 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 -- 172.21.15.77:0/215642530 >> 172.21.15.77:0/215642530 conn(0x564a00491210 msgr2=0x564a00558d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 -- 172.21.15.77:0/215642530 shutdown_connections 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 1 -- 172.21.15.77:0/215642530 wait complete. 2024-05-27T08:47:05.440 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.431+0000 7f7857b2fb80 20 client.15741 trim_cache size 0 max 16384 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.451+0000 7f7857b2fb80 1 Processor -- start 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.451+0000 7f7857b2fb80 1 -- start start 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a0056f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00565e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00566360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78481addb0 con 0x564a00563970 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78481af9b0 con 0x564a00564330 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78481ae480 con 0x564a00569340 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a0056f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.453 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a0056f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59460/0 (socket says 172.21.15.77:59460) 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 -- 172.21.15.77:0/1773757471 learned_addr learned my addr 172.21.15.77:0/1773757471 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f783e7fc700 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00565e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f78477fe700 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00566360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 -- 172.21.15.77:0/1773757471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 msgr2=0x564a00566360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00566360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 -- 172.21.15.77:0/1773757471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 msgr2=0x564a00565e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00565e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 -- 172.21.15.77:0/1773757471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a00569cf0 con 0x564a00563970 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f78477fe700 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00566360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f783e7fc700 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00565e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f7846ffd700 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a0056f8c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x564a005707d0 tx=0x7f7838013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3eedabadcfc02e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:05.454 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f783cff9700 1 -- 172.21.15.77:0/1773757471 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838015860 con 0x564a00563970 2024-05-27T08:47:05.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f783cff9700 1 -- 172.21.15.77:0/1773757471 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78380282c0 con 0x564a00563970 2024-05-27T08:47:05.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.452+0000 7f783cff9700 1 -- 172.21.15.77:0/1773757471 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838021420 con 0x564a00563970 2024-05-27T08:47:05.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 -- 172.21.15.77:0/1773757471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 msgr2=0x564a0056f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a0056f8c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x564a005707d0 tx=0x7f7838013000 comp rx=0 tx=0).stop 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 -- 172.21.15.77:0/1773757471 shutdown_connections 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00569340 0x564a00566360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a0056f8c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/1773757471 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00564330 0x564a00565e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 -- 172.21.15.77:0/1773757471 >> 172.21.15.77:0/1773757471 conn(0x564a00491210 msgr2=0x564a005631f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 -- 172.21.15.77:0/1773757471 shutdown_connections 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.453+0000 7f7857b2fb80 1 -- 172.21.15.77:0/1773757471 wait complete. 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 1 -- start start 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 1 Processor -- start 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00582c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00564330 0x564a0057f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00569340 0x564a0057fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f78481af220 con 0x564a00563970 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f78481aeb50 con 0x564a00569340 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78481abd70 con 0x564a00564330 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7857b2fb80 10 client.0 _ll_register_callbacks cb 0x564a0044dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00582c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00582c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:59472/0 (socket says 172.21.15.77:59472) 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 -- 172.21.15.77:0/3177538843 learned_addr learned my addr 172.21.15.77:0/3177538843 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f783e7fc700 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00564330 0x564a0057f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 -- 172.21.15.77:0/3177538843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00564330 msgr2=0x564a0057f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00564330 0x564a0057f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 -- 172.21.15.77:0/3177538843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00569340 msgr2=0x564a0057fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:05.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00569340 0x564a0057fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 -- 172.21.15.77:0/3177538843 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a00580240 con 0x564a00563970 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f7846ffd700 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00582c40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7838028ec0 tx=0x7f7838028cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31ad818a25a5933 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.454+0000 7f782e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f782e7fc700 1 -- 172.21.15.77:0/3177538843 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78380288c0 con 0x564a00563970 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f782e7fc700 1 -- 172.21.15.77:0/3177538843 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8586+0+0 (secure 0 0 0) 0x7f7838015860 con 0x564a00563970 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 1 -- 172.21.15.77:0/3177538843 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78380282c0 con 0x564a00563970 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f782e7fc700 1 -- 172.21.15.77:0/3177538843 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7838015a60 con 0x564a00563970 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 10 client.15747 fetch_fsmap learned FSMap version 37 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 1 -- 172.21.15.77:0/3177538843 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564a005ecdc0 con 0x564a00563970 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f782e7fc700 1 -- 172.21.15.77:0/3177538843 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78380282c0 con 0x564a00563970 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f782e7fc700 1 -- 172.21.15.77:0/3177538843 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 37) ==== 8370+0+0 (secure 0 0 0) 0x7f78380062f0 con 0x564a00563970 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 10 client.15747 fetch_fsmap finished waiting for FSMap version 37 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 10 client.15747 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:47:05.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 20 client.15747 populate_metadata read hostname 'smithi077' 2024-05-27T08:47:05.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564a005ed570 0x564a005ef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:05.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 4 client.15747 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.3, "await_for": 45.0}] 2024-05-27T08:47:05.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.455+0000 7f7857b2fb80 1 -- 172.21.15.77:0/3177538843 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.3, "await_for": 45.0}) -- 0x564a004c4d70 con 0x564a005ed570 2024-05-27T08:47:05.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.456+0000 7f783e7fc700 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564a005ed570 0x564a005ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:05.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.456+0000 7f782e7fc700 10 client.15747 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:47:05.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.456+0000 7f783e7fc700 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564a005ed570 0x564a005ef990 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be1ef738cd8a652 server_cookie=91419af38f43c56d in_seq=0 out_seq=0 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f782e7fc700 1 -- 172.21.15.77:0/3177538843 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564a004c4d70 con 0x564a005ed570 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f782e7fc700 10 client.15747 handle_command_reply: tid=0 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 client.15747 shutdown 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 -- 172.21.15.77:0/3177538843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 msgr2=0x564a00582c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00582c40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7838028ec0 tx=0x7f7838028cf0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 -- 172.21.15.77:0/3177538843 shutdown_connections 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564a005ed570 0x564a005ef990 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564a00569340 0x564a0057fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.529 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564a00563970 0x564a00582c40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 --2- 172.21.15.77:0/3177538843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a00564330 0x564a0057f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f782e7fc700 0 client.15747 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 -- 172.21.15.77:0/3177538843 >> 172.21.15.77:0/3177538843 conn(0x564a00491210 msgr2=0x564a005584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 -- 172.21.15.77:0/3177538843 shutdown_connections 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 1 -- 172.21.15.77:0/3177538843 wait complete. 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.528+0000 7f7857b2fb80 20 client.15747 trim_cache size 0 max 16384 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 37, 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:47:05.530 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 26, 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "e4a2ab5f": { 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 26, 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 144.3, 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:47:05.531 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:47:05.532 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "QUIESCED", 2024-05-27T08:47:05.532 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:47:05.532 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:05.532 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:05.532 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:05.532 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:05.532 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:05.532 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 -- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f78340b2b20 msgr2=0x7f78340b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f78340b2b20 0x7f78340b4fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7848107c50 tx=0x7f7838009190 comp rx=0 tx=0).stop 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 -- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848108840 msgr2=0x7f7848199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848108840 0x7f7848199a50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7840007ee0 tx=0x7f784000d3b0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 -- 172.21.15.77:0/1705596270 shutdown_connections 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f78340b2b20 0x7f78340b4fe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7848108840 0x7f7848199a50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7848103850 0x7f784819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 --2- 172.21.15.77:0/1705596270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7848104200 0x7f784819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 -- 172.21.15.77:0/1705596270 >> 172.21.15.77:0/1705596270 conn(0x7f784807a610 msgr2=0x7f7848102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 -- 172.21.15.77:0/1705596270 shutdown_connections 2024-05-27T08:47:05.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:05.529+0000 7f784d299700 1 -- 172.21.15.77:0/1705596270 wait complete. 2024-05-27T08:47:05.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.92 MB/sec execute 299 sec latency 2288.077 ms 2024-05-27T08:47:05.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:05 smithi092 ceph-mon[36049]: pgmap v272: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-05-27T08:47:06.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:05 smithi179 ceph-mon[36481]: pgmap v272: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-05-27T08:47:06.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:05 smithi077 ceph-mon[32007]: pgmap v272: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-05-27T08:47:06.165 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e4a2ab5f', quiesce took 5.9 seconds. Will release after: 18.4 2024-05-27T08:47:06.612 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.90 MB/sec execute 300 sec latency 1132.075 ms 2024-05-27T08:47:07.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.89 MB/sec execute 301 sec latency 2132.171 ms 2024-05-27T08:47:07.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:07 smithi092 ceph-mon[36049]: pgmap v273: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-05-27T08:47:08.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:07 smithi179 ceph-mon[36481]: pgmap v273: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-05-27T08:47:08.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:07 smithi077 ceph-mon[32007]: pgmap v273: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-05-27T08:47:08.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.88 MB/sec execute 302 sec latency 3132.269 ms 2024-05-27T08:47:09.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.87 MB/sec execute 303 sec latency 4132.388 ms 2024-05-27T08:47:09.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:09 smithi092 ceph-mon[36049]: pgmap v274: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-27T08:47:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:09 smithi179 ceph-mon[36481]: pgmap v274: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-27T08:47:10.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:09 smithi077 ceph-mon[32007]: pgmap v274: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-27T08:47:10.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.85 MB/sec execute 304 sec latency 5132.470 ms 2024-05-27T08:47:11.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.84 MB/sec execute 305 sec latency 6132.529 ms 2024-05-27T08:47:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:11 smithi077 ceph-mon[32007]: pgmap v275: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-27T08:47:11.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:11 smithi077 ceph-mon[32007]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T08:47:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:11 smithi092 ceph-mon[36049]: pgmap v275: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-27T08:47:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:11 smithi092 ceph-mon[36049]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T08:47:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:11 smithi179 ceph-mon[36481]: pgmap v275: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-05-27T08:47:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:11 smithi179 ceph-mon[36481]: osdmap e92: 12 total, 12 up, 12 in 2024-05-27T08:47:12.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.83 MB/sec execute 306 sec latency 7132.653 ms 2024-05-27T08:47:13.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.82 MB/sec execute 307 sec latency 8132.729 ms 2024-05-27T08:47:13.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:13 smithi077 ceph-mon[32007]: pgmap v277: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-05-27T08:47:13.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:13 smithi092 ceph-mon[36049]: pgmap v277: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-05-27T08:47:14.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:13 smithi179 ceph-mon[36481]: pgmap v277: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-05-27T08:47:14.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.80 MB/sec execute 308 sec latency 9132.854 ms 2024-05-27T08:47:15.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.79 MB/sec execute 309 sec latency 10132.973 ms 2024-05-27T08:47:15.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:15 smithi077 ceph-mon[32007]: pgmap v278: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 970 KiB/s wr, 73 op/s 2024-05-27T08:47:15.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:15 smithi092 ceph-mon[36049]: pgmap v278: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 970 KiB/s wr, 73 op/s 2024-05-27T08:47:16.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:15 smithi179 ceph-mon[36481]: pgmap v278: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 970 KiB/s wr, 73 op/s 2024-05-27T08:47:16.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.78 MB/sec execute 310 sec latency 11133.027 ms 2024-05-27T08:47:17.613 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.77 MB/sec execute 311 sec latency 12133.145 ms 2024-05-27T08:47:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:17 smithi092 ceph-mon[36049]: pgmap v279: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 970 KiB/s wr, 73 op/s 2024-05-27T08:47:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:17 smithi179 ceph-mon[36481]: pgmap v279: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 970 KiB/s wr, 73 op/s 2024-05-27T08:47:18.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:17 smithi077 ceph-mon[32007]: pgmap v279: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 970 KiB/s wr, 73 op/s 2024-05-27T08:47:18.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.75 MB/sec execute 312 sec latency 13133.239 ms 2024-05-27T08:47:19.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.74 MB/sec execute 313 sec latency 14133.345 ms 2024-05-27T08:47:19.931 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:19 smithi077 ceph-mon[32007]: pgmap v280: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 770 KiB/s wr, 47 op/s 2024-05-27T08:47:19.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:19 smithi092 ceph-mon[36049]: pgmap v280: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 770 KiB/s wr, 47 op/s 2024-05-27T08:47:20.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:19 smithi179 ceph-mon[36481]: pgmap v280: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 770 KiB/s wr, 47 op/s 2024-05-27T08:47:20.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.73 MB/sec execute 314 sec latency 15133.422 ms 2024-05-27T08:47:21.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.72 MB/sec execute 315 sec latency 16133.455 ms 2024-05-27T08:47:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:21 smithi092 ceph-mon[36049]: pgmap v281: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 770 KiB/s wr, 47 op/s 2024-05-27T08:47:22.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:21 smithi179 ceph-mon[36481]: pgmap v281: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 770 KiB/s wr, 47 op/s 2024-05-27T08:47:22.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:21 smithi077 ceph-mon[32007]: pgmap v281: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 770 KiB/s wr, 47 op/s 2024-05-27T08:47:22.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.71 MB/sec execute 316 sec latency 17133.550 ms 2024-05-27T08:47:23.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.69 MB/sec execute 317 sec latency 18133.663 ms 2024-05-27T08:47:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:23 smithi092 ceph-mon[36049]: pgmap v282: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 943 KiB/s wr, 59 op/s 2024-05-27T08:47:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:23 smithi179 ceph-mon[36481]: pgmap v282: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 943 KiB/s wr, 59 op/s 2024-05-27T08:47:24.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:23 smithi077 ceph-mon[32007]: pgmap v282: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 943 KiB/s wr, 59 op/s 2024-05-27T08:47:24.566 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:47:24.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.68 MB/sec execute 318 sec latency 19133.711 ms 2024-05-27T08:47:24.826 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:47:25.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.67 MB/sec execute 319 sec latency 20133.792 ms 2024-05-27T08:47:25.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:25 smithi092 ceph-mon[36049]: pgmap v283: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 571 KiB/s wr, 39 op/s 2024-05-27T08:47:25.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:25 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:47:26.070 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:25 smithi077 ceph-mon[32007]: pgmap v283: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 571 KiB/s wr, 39 op/s 2024-05-27T08:47:26.071 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:25 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:47:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:25 smithi179 ceph-mon[36481]: pgmap v283: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 571 KiB/s wr, 39 op/s 2024-05-27T08:47:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:25 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:47:26.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2953674744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4102df0 msgr2=0x7fc2b410f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:26.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2953674744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4102df0 0x7fc2b410f2e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00b0a0 tx=0x7fc2ac00b3b0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2953674744 shutdown_connections 2024-05-27T08:47:26.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2953674744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4102df0 0x7fc2b410f2e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2953674744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b4108510 0x7fc2b41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2953674744 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b4102430 0x7fc2b41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2953674744 >> 172.21.15.77:0/2953674744 conn(0x7fc2b40793b0 msgr2=0x7fc2b40797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:26.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2953674744 shutdown_connections 2024-05-27T08:47:26.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2953674744 wait complete. 2024-05-27T08:47:26.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.593+0000 7fc2bc5cd700 1 Processor -- start 2024-05-27T08:47:26.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bc5cd700 1 -- start start 2024-05-27T08:47:26.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bc5cd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b4102430 0x7fc2b419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:26.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bc5cd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b4102df0 0x7fc2b419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:26.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bc5cd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4108510 0x7fc2b4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:26.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bc5cd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41125e0 con 0x7fc2b4108510 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bc5cd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4112460 con 0x7fc2b4102430 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bc5cd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4112760 con 0x7fc2b4102df0 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4108510 0x7fc2b4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2ba369700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b4102430 0x7fc2b419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4108510 0x7fc2b4198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49902/0 (socket says 172.21.15.77:49902) 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 -- 172.21.15.77:0/2288831779 learned_addr learned my addr 172.21.15.77:0/2288831779 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2b9b68700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b4102df0 0x7fc2b419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 -- 172.21.15.77:0/2288831779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b4102df0 msgr2=0x7fc2b419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b4102df0 0x7fc2b419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 -- 172.21.15.77:0/2288831779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b4102430 msgr2=0x7fc2b419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:26.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b4102430 0x7fc2b419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 -- 172.21.15.77:0/2288831779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b4198fe0 con 0x7fc2b4108510 2024-05-27T08:47:26.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.594+0000 7fc2bab6a700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4108510 0x7fc2b4198880 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00b830 tx=0x7fc2ac004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:26.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.595+0000 7fc2ab7fe700 1 -- 172.21.15.77:0/2288831779 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac021070 con 0x7fc2b4108510 2024-05-27T08:47:26.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.595+0000 7fc2ab7fe700 1 -- 172.21.15.77:0/2288831779 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2ac007e30 con 0x7fc2b4108510 2024-05-27T08:47:26.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.595+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b4199270 con 0x7fc2b4108510 2024-05-27T08:47:26.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.595+0000 7fc2ab7fe700 1 -- 172.21.15.77:0/2288831779 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac004700 con 0x7fc2b4108510 2024-05-27T08:47:26.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.595+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b406bab0 con 0x7fc2b4108510 2024-05-27T08:47:26.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.596+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b406a230 con 0x7fc2b4108510 2024-05-27T08:47:26.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.597+0000 7fc2ab7fe700 1 -- 172.21.15.77:0/2288831779 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc2ac003d70 con 0x7fc2b4108510 2024-05-27T08:47:26.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.598+0000 7fc2ab7fe700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fc2a00b29f0 0x7fc2a00b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:26.600 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.598+0000 7fc2ba369700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fc2a00b29f0 0x7fc2a00b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:26.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.598+0000 7fc2ab7fe700 1 -- 172.21.15.77:0/2288831779 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fc2ac00f020 con 0x7fc2b4108510 2024-05-27T08:47:26.601 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.598+0000 7fc2ba369700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fc2a00b29f0 0x7fc2a00b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2b41999c0 tx=0x7fc2b000d5d0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:26.602 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.601+0000 7fc2ab7fe700 1 -- 172.21.15.77:0/2288831779 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2ac02a020 con 0x7fc2b4108510 2024-05-27T08:47:26.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.66 MB/sec execute 320 sec latency 21133.835 ms 2024-05-27T08:47:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.860+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2b406c450 con 0x7fc2b4108510 2024-05-27T08:47:26.862 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.861+0000 7fc2ab7fe700 1 -- 172.21.15.77:0/2288831779 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11928 (secure 0 0 0) 0x7fc2ac0ab960 con 0x7fc2b4108510 2024-05-27T08:47:26.863 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:47:26.863 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:47:23.677029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24412,24433,24410,24434,24418]},"id":1}]} 2024-05-27T08:47:26.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fc2a00b29f0 msgr2=0x7fc2a00b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:26.864 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fc2a00b29f0 0x7fc2a00b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2b41999c0 tx=0x7fc2b000d5d0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4108510 msgr2=0x7fc2b4198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4108510 0x7fc2b4198880 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac00b830 tx=0x7fc2ac004ce0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 shutdown_connections 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fc2a00b29f0 0x7fc2a00b4eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2b4108510 0x7fc2b4198880 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc2b4102430 0x7fc2b419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 --2- 172.21.15.77:0/2288831779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2b4102df0 0x7fc2b419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 >> 172.21.15.77:0/2288831779 conn(0x7fc2b40793b0 msgr2=0x7fc2b4100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 shutdown_connections 2024-05-27T08:47:26.865 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:26.863+0000 7fc2bc5cd700 1 -- 172.21.15.77:0/2288831779 wait complete. 2024-05-27T08:47:26.866 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 38 2024-05-27T08:47:27.465 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id e4a2ab5f --release --await' 2024-05-27T08:47:27.465 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db --set-id e4a2ab5f --release --await 2024-05-27T08:47:27.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.65 MB/sec execute 321 sec latency 22133.890 ms 2024-05-27T08:47:27.705 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:47:27.827 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:27 smithi077 ceph-mon[32007]: pgmap v284: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 571 KiB/s wr, 39 op/s 2024-05-27T08:47:27.828 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:27 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2288831779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:47:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:27 smithi092 ceph-mon[36049]: pgmap v284: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 571 KiB/s wr, 39 op/s 2024-05-27T08:47:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:27 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2288831779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:47:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:27 smithi179 ceph-mon[36481]: pgmap v284: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 571 KiB/s wr, 39 op/s 2024-05-27T08:47:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:27 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2288831779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:47:28.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.64 MB/sec execute 322 sec latency 23133.927 ms 2024-05-27T08:47:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 -- 172.21.15.77:0/347355821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30102df0 msgr2=0x7fab3010f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 --2- 172.21.15.77:0/347355821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30102df0 0x7fab3010f2e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fab18009990 tx=0x7fab18009ca0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 -- 172.21.15.77:0/347355821 shutdown_connections 2024-05-27T08:47:29.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 --2- 172.21.15.77:0/347355821 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30102df0 0x7fab3010f2e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 --2- 172.21.15.77:0/347355821 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab30108510 0x7fab301088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 --2- 172.21.15.77:0/347355821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab30102430 0x7fab301028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 -- 172.21.15.77:0/347355821 >> 172.21.15.77:0/347355821 conn(0x7fab300793b0 msgr2=0x7fab300797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 -- 172.21.15.77:0/347355821 shutdown_connections 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.608+0000 7fab36b7e700 1 -- 172.21.15.77:0/347355821 wait complete. 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab36b7e700 1 Processor -- start 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab36b7e700 1 -- start start 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab36b7e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab30102430 0x7fab3019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab36b7e700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab30102df0 0x7fab3019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab36b7e700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30108510 0x7fab30199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab36b7e700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab301125c0 con 0x7fab30108510 2024-05-27T08:47:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab36b7e700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112440 con 0x7fab30102df0 2024-05-27T08:47:29.612 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab36b7e700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112740 con 0x7fab30102430 2024-05-27T08:47:29.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab3511b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30108510 0x7fab30199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.613 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab3491a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab30102430 0x7fab3019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab3511b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30108510 0x7fab30199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41198/0 (socket says 172.21.15.77:41198) 2024-05-27T08:47:29.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab3511b700 1 -- 172.21.15.77:0/2620265276 learned_addr learned my addr 172.21.15.77:0/2620265276 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:29.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab3491a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab30102430 0x7fab3019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.77:50360/0 (socket says 172.21.15.77:50360) 2024-05-27T08:47:29.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.609+0000 7fab2ffff700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab30102df0 0x7fab3019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab3511b700 1 -- 172.21.15.77:0/2620265276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab30102430 msgr2=0x7fab3019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab3511b700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab30102430 0x7fab3019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab3511b700 1 -- 172.21.15.77:0/2620265276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab30102df0 msgr2=0x7fab3019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab3511b700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab30102df0 0x7fab3019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab3511b700 1 -- 172.21.15.77:0/2620265276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3019a0e0 con 0x7fab30108510 2024-05-27T08:47:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab3511b700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30108510 0x7fab30199980 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fab180047b0 tx=0x7fab18005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab2dffb700 1 -- 172.21.15.77:0/2620265276 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18026070 con 0x7fab30108510 2024-05-27T08:47:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab2dffb700 1 -- 172.21.15.77:0/2620265276 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab18004c90 con 0x7fab30108510 2024-05-27T08:47:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab36b7e700 1 -- 172.21.15.77:0/2620265276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3019a370 con 0x7fab30108510 2024-05-27T08:47:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab2dffb700 1 -- 172.21.15.77:0/2620265276 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18003cd0 con 0x7fab30108510 2024-05-27T08:47:29.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.610+0000 7fab36b7e700 1 -- 172.21.15.77:0/2620265276 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab301a6690 con 0x7fab30108510 2024-05-27T08:47:29.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.613+0000 7fab2dffb700 1 -- 172.21.15.77:0/2620265276 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fab18029070 con 0x7fab30108510 2024-05-27T08:47:29.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 306760 3.63 MB/sec execute 323 sec latency 24133.968 ms 2024-05-27T08:47:29.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.613+0000 7fab2dffb700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fab1c0b29f0 0x7fab1c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab3491a700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fab1c0b29f0 0x7fab1c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab2dffb700 1 -- 172.21.15.77:0/2620265276 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fab180e2e20 con 0x7fab30108510 2024-05-27T08:47:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab41414b80 1 Processor -- start 2024-05-27T08:47:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab41414b80 1 -- start start 2024-05-27T08:47:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab3491a700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fab1c0b29f0 0x7fab1c0b4eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab28009550 tx=0x7fab280091f0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab41414b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a5822b0 0x555a8a582690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab41414b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 0x555a8a57cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab41414b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a57d2a0 0x555a8a5890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab41414b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a8a4dbd90 con 0x555a8a5822b0 2024-05-27T08:47:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab41414b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555a8a4cb930 con 0x555a8a57c8e0 2024-05-27T08:47:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab41414b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555a8a4cf970 con 0x555a8a57d2a0 2024-05-27T08:47:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab3491a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a5822b0 0x555a8a582690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab3491a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a5822b0 0x555a8a582690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41202/0 (socket says 172.21.15.77:41202) 2024-05-27T08:47:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab3491a700 1 -- 172.21.15.77:0/330728710 learned_addr learned my addr 172.21.15.77:0/330728710 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:29.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.614+0000 7fab2ffff700 1 --2- 172.21.15.77:0/330728710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 0x555a8a57cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab3491a700 1 -- 172.21.15.77:0/330728710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a57d2a0 msgr2=0x555a8a5890e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab3491a700 1 --2- 172.21.15.77:0/330728710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a57d2a0 0x555a8a5890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab3491a700 1 -- 172.21.15.77:0/330728710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 msgr2=0x555a8a57cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab3491a700 1 --2- 172.21.15.77:0/330728710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 0x555a8a57cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab3491a700 1 -- 172.21.15.77:0/330728710 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8a41b9d0 con 0x555a8a5822b0 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab3491a700 1 --2- 172.21.15.77:0/330728710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a5822b0 0x555a8a582690 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fab28009650 tx=0x7fab28013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0551d230686172e server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab26ffd700 1 -- 172.21.15.77:0/330728710 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2801c050 con 0x555a8a5822b0 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab26ffd700 1 -- 172.21.15.77:0/330728710 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28026070 con 0x555a8a5822b0 2024-05-27T08:47:29.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.615+0000 7fab26ffd700 1 -- 172.21.15.77:0/330728710 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab280156a0 con 0x555a8a5822b0 2024-05-27T08:47:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 -- 172.21.15.77:0/330728710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a5822b0 msgr2=0x555a8a582690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 --2- 172.21.15.77:0/330728710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a5822b0 0x555a8a582690 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fab28009650 tx=0x7fab28013000 comp rx=0 tx=0).stop 2024-05-27T08:47:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 -- 172.21.15.77:0/330728710 shutdown_connections 2024-05-27T08:47:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 --2- 172.21.15.77:0/330728710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a57d2a0 0x555a8a5890e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 --2- 172.21.15.77:0/330728710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a5822b0 0x555a8a582690 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 --2- 172.21.15.77:0/330728710 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 0x555a8a57cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 -- 172.21.15.77:0/330728710 >> 172.21.15.77:0/330728710 conn(0x555a8a4aa180 msgr2=0x555a8a5771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:29.626 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 -- 172.21.15.77:0/330728710 shutdown_connections 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 -- 172.21.15.77:0/330728710 wait complete. 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.616+0000 7fab41414b80 1 -- start start 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab41414b80 1 Processor -- start 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab41414b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a588e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab41414b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a586110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab41414b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a586650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab41414b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a8a3d20a0 con 0x555a8a57c8e0 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab41414b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x555a8a46d8c0 con 0x555a8a57d2a0 2024-05-27T08:47:29.627 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab41414b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555a8a589680 con 0x555a8a5822b0 2024-05-27T08:47:29.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab41414b80 10 client.0 _ll_register_callbacks cb 0x555a8a466a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:47:29.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3491a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a588e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3491a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a588e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41210/0 (socket says 172.21.15.77:41210) 2024-05-27T08:47:29.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3491a700 1 -- 172.21.15.77:0/2349467353 learned_addr learned my addr 172.21.15.77:0/2349467353 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:29.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab2ffff700 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a586110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3511b700 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a586650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3491a700 1 -- 172.21.15.77:0/2349467353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 msgr2=0x555a8a586650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:29.628 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3491a700 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a586650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3491a700 1 -- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 msgr2=0x555a8a586110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3491a700 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a586110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab3491a700 1 -- 172.21.15.77:0/2349467353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8a46a790 con 0x555a8a57c8e0 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.617+0000 7fab2ffff700 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a586110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab3491a700 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a588e10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fab280058f0 tx=0x7fab28005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d80c33a8b58fe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab0ffff700 1 -- 172.21.15.77:0/2349467353 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2802f070 con 0x555a8a57c8e0 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab0ffff700 1 -- 172.21.15.77:0/2349467353 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fab28034460 con 0x555a8a57c8e0 2024-05-27T08:47:29.629 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab41414b80 1 -- 172.21.15.77:0/2349467353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab28026070 con 0x555a8a57c8e0 2024-05-27T08:47:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab0ffff700 1 -- 172.21.15.77:0/2349467353 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fab28034660 con 0x555a8a57c8e0 2024-05-27T08:47:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab41414b80 10 client.15765 fetch_fsmap learned FSMap version 38 2024-05-27T08:47:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab41414b80 1 -- 172.21.15.77:0/2349467353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555a8a593e30 con 0x555a8a57c8e0 2024-05-27T08:47:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab0ffff700 1 -- 172.21.15.77:0/2349467353 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28026070 con 0x555a8a57c8e0 2024-05-27T08:47:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab0ffff700 1 -- 172.21.15.77:0/2349467353 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fab280157d0 con 0x555a8a57c8e0 2024-05-27T08:47:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab41414b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 38 2024-05-27T08:47:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab41414b80 10 client.15765 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:47:29.630 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.618+0000 7fab41414b80 20 client.15765 populate_metadata read hostname 'smithi077' 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.619+0000 7fab41414b80 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x555a8a596410 0x555a8a598800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.619+0000 7fab41414b80 4 client.15765 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.619+0000 7fab41414b80 1 -- 172.21.15.77:0/2349467353 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a8a548ae0 con 0x555a8a596410 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.619+0000 7fab2ffff700 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x555a8a596410 0x555a8a598800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.619+0000 7fab2ffff700 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x555a8a596410 0x555a8a598800 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff791554cf1226f server_cookie=c90cf384be5e9084 in_seq=0 out_seq=0 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.619+0000 7fab0ffff700 10 client.15765 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab0ffff700 1 -- 172.21.15.77:0/2349467353 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a8a548ae0 con 0x555a8a596410 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab0ffff700 10 client.15765 handle_command_reply: tid=0 2024-05-27T08:47:29.631 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 client.15765 shutdown 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 -- 172.21.15.77:0/2349467353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 msgr2=0x555a8a588e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a588e10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fab280058f0 tx=0x7fab28005920 comp rx=0 tx=0).stop 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 -- 172.21.15.77:0/2349467353 shutdown_connections 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x555a8a596410 0x555a8a598800 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a586650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a588e10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 --2- 172.21.15.77:0/2349467353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a586110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 -- 172.21.15.77:0/2349467353 >> 172.21.15.77:0/2349467353 conn(0x555a8a4aa180 msgr2=0x555a8a571ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab0ffff700 0 client.15765 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:47:29.632 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 -- 172.21.15.77:0/2349467353 shutdown_connections 2024-05-27T08:47:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 1 -- 172.21.15.77:0/2349467353 wait complete. 2024-05-27T08:47:29.633 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.620+0000 7fab41414b80 20 client.15765 trim_cache size 0 max 16384 2024-05-27T08:47:29.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab41414b80 1 Processor -- start 2024-05-27T08:47:29.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab41414b80 1 -- start start 2024-05-27T08:47:29.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab41414b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 0x555a8a5804a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab41414b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57d2a0 0x555a8a588950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.643 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab41414b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a591910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab41414b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab301abae0 con 0x555a8a57d2a0 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab41414b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ad650 con 0x555a8a57c8e0 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab41414b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ac1b0 con 0x555a8a5822b0 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab2ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57d2a0 0x555a8a588950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab2ffff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57d2a0 0x555a8a588950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41222/0 (socket says 172.21.15.77:41222) 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab2ffff700 1 -- 172.21.15.77:0/3686466924 learned_addr learned my addr 172.21.15.77:0/3686466924 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab3511b700 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a591910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab3491a700 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 0x555a8a5804a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab2ffff700 1 -- 172.21.15.77:0/3686466924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 msgr2=0x555a8a591910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab2ffff700 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a591910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.641+0000 7fab2ffff700 1 -- 172.21.15.77:0/3686466924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 msgr2=0x555a8a5804a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:29.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.642+0000 7fab2ffff700 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 0x555a8a5804a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.642+0000 7fab2ffff700 1 -- 172.21.15.77:0/3686466924 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8a587050 con 0x555a8a57d2a0 2024-05-27T08:47:29.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.642+0000 7fab2ffff700 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57d2a0 0x555a8a588950 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fab30079ab0 tx=0x7fab2000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecb34fec2dce47c server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:29.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.642+0000 7fab267fc700 1 -- 172.21.15.77:0/3686466924 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20003a80 con 0x555a8a57d2a0 2024-05-27T08:47:29.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.642+0000 7fab267fc700 1 -- 172.21.15.77:0/3686466924 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab20004ce0 con 0x555a8a57d2a0 2024-05-27T08:47:29.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.642+0000 7fab267fc700 1 -- 172.21.15.77:0/3686466924 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab200055e0 con 0x555a8a57d2a0 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 -- 172.21.15.77:0/3686466924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57d2a0 msgr2=0x555a8a588950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57d2a0 0x555a8a588950 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fab30079ab0 tx=0x7fab2000bc20 comp rx=0 tx=0).stop 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 -- 172.21.15.77:0/3686466924 shutdown_connections 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a591910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57c8e0 0x555a8a5804a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 --2- 172.21.15.77:0/3686466924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57d2a0 0x555a8a588950 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 -- 172.21.15.77:0/3686466924 >> 172.21.15.77:0/3686466924 conn(0x555a8a4aa180 msgr2=0x555a8a57fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 -- 172.21.15.77:0/3686466924 shutdown_connections 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 -- 172.21.15.77:0/3686466924 wait complete. 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 -- start start 2024-05-27T08:47:29.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.643+0000 7fab41414b80 1 Processor -- start 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab41414b80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a59bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab41414b80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a59a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab41414b80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a59a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab41414b80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab301acf50 con 0x555a8a57c8e0 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab41414b80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ac880 con 0x555a8a57d2a0 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab41414b80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fab301a9aa0 con 0x555a8a5822b0 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab41414b80 10 client.0 _ll_register_callbacks cb 0x555a8a466a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a59bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a59bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:41236/0 (socket says 172.21.15.77:41236) 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 -- 172.21.15.77:0/1296086061 learned_addr learned my addr 172.21.15.77:0/1296086061 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3511b700 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a59a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab2ffff700 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a59a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 -- 172.21.15.77:0/1296086061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 msgr2=0x555a8a59a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a59a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 -- 172.21.15.77:0/1296086061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 msgr2=0x555a8a59a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a59a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 -- 172.21.15.77:0/1296086061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8a59ae10 con 0x555a8a57c8e0 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab3491a700 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a59bc60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab28015c70 tx=0x7fab28003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488b24522bcfc6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab0ffff700 1 -- 172.21.15.77:0/1296086061 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28041070 con 0x555a8a57c8e0 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.644+0000 7fab41414b80 1 -- 172.21.15.77:0/1296086061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab20004ce0 con 0x555a8a57c8e0 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab0ffff700 1 -- 172.21.15.77:0/1296086061 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fab28015440 con 0x555a8a57c8e0 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab0ffff700 1 -- 172.21.15.77:0/1296086061 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fab2801c070 con 0x555a8a57c8e0 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab41414b80 10 client.15771 fetch_fsmap learned FSMap version 38 2024-05-27T08:47:29.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab41414b80 1 -- 172.21.15.77:0/1296086061 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555a8a601ea0 con 0x555a8a57c8e0 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab0ffff700 1 -- 172.21.15.77:0/1296086061 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab20004ce0 con 0x555a8a57c8e0 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab0ffff700 1 -- 172.21.15.77:0/1296086061 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7fab2802b6b0 con 0x555a8a57c8e0 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab41414b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 38 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab41414b80 10 client.15771 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab41414b80 20 client.15771 populate_metadata read hostname 'smithi077' 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab41414b80 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x555a8a602520 0x555a8a604940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab41414b80 4 client.15771 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e4a2ab5f", "release": true, "await": true}] 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab41414b80 1 -- 172.21.15.77:0/1296086061 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e4a2ab5f", "release": true, "await": true}) -- 0x555a8a4ddce0 con 0x555a8a602520 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.645+0000 7fab2ffff700 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x555a8a602520 0x555a8a604940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.646+0000 7fab0ffff700 10 client.15771 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:47:29.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.646+0000 7fab2ffff700 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x555a8a602520 0x555a8a604940 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6ba04e4a24e0ff server_cookie=29196ee5651a2c4c in_seq=0 out_seq=0 2024-05-27T08:47:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.655+0000 7fab0ffff700 1 -- 172.21.15.77:0/1296086061 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555a8a4ddce0 con 0x555a8a602520 2024-05-27T08:47:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.655+0000 7fab0ffff700 10 client.15771 handle_command_reply: tid=0 2024-05-27T08:47:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.655+0000 7fab41414b80 1 client.15771 shutdown 2024-05-27T08:47:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 -- 172.21.15.77:0/1296086061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 msgr2=0x555a8a59bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a59bc60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab28015c70 tx=0x7fab28003bb0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 -- 172.21.15.77:0/1296086061 shutdown_connections 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x555a8a602520 0x555a8a604940 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a8a5822b0 0x555a8a59a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a8a57c8e0 0x555a8a59bc60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 --2- 172.21.15.77:0/1296086061 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x555a8a57d2a0 0x555a8a59a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab0ffff700 0 client.15771 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 -- 172.21.15.77:0/1296086061 >> 172.21.15.77:0/1296086061 conn(0x555a8a4aa180 msgr2=0x555a8a580620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 -- 172.21.15.77:0/1296086061 shutdown_connections 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 1 -- 172.21.15.77:0/1296086061 wait complete. 2024-05-27T08:47:29.658 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.656+0000 7fab41414b80 20 client.15771 trim_cache size 0 max 16384 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 38, 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 28, 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": { 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "e4a2ab5f": { 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "version": 28, 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "age_ref": 0.0, 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:47:29.659 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: "timeout": 90.0, 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: "expiration": 144.3, 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: "members": { 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: "file:/": { 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: "excluded": false, 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: "state": { 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RELEASED", 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: "age": 0.0 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:29.660 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:47:29.661 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:47:29.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 -- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fab1c0b29f0 msgr2=0x7fab1c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fab1c0b29f0 0x7fab1c0b4eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fab28009550 tx=0x7fab280091f0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 -- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30108510 msgr2=0x7fab30199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:47:29.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30108510 0x7fab30199980 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fab180047b0 tx=0x7fab18005e80 comp rx=0 tx=0).stop 2024-05-27T08:47:29.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 -- 172.21.15.77:0/2620265276 shutdown_connections 2024-05-27T08:47:29.661 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fab1c0b29f0 0x7fab1c0b4eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab30108510 0x7fab30199980 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab30102430 0x7fab3019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 --2- 172.21.15.77:0/2620265276 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fab30102df0 0x7fab3019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:47:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 -- 172.21.15.77:0/2620265276 >> 172.21.15.77:0/2620265276 conn(0x7fab300793b0 msgr2=0x7fab3006bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:47:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 -- 172.21.15.77:0/2620265276 shutdown_connections 2024-05-27T08:47:29.662 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:47:29.657+0000 7fab36b7e700 1 -- 172.21.15.77:0/2620265276 wait complete. 2024-05-27T08:47:29.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:29 smithi077 ceph-mon[32007]: pgmap v285: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 833 KiB/s wr, 56 op/s 2024-05-27T08:47:29.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:29 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:47:29.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:29 smithi092 ceph-mon[36049]: pgmap v285: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 833 KiB/s wr, 56 op/s 2024-05-27T08:47:29.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:29 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:47:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:29 smithi179 ceph-mon[36481]: pgmap v285: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 833 KiB/s wr, 56 op/s 2024-05-27T08:47:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:29 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:47:30.363 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e4a2ab5f', total seconds elapsed: 30.1 2024-05-27T08:47:30.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 308832 3.66 MB/sec execute 324 sec latency 24177.891 ms 2024-05-27T08:47:30.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:30 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:47:30.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:30 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:47:30.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:30 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:47:31.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:30 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:47:31.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:30 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:47:31.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:30 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:47:31.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:30 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:47:31.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:30 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:47:31.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:30 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:47:31.614 INFO:tasks.workunit.client.0.smithi077.stdout: 1 310119 3.66 MB/sec execute 325 sec latency 15.997 ms 2024-05-27T08:47:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:31 smithi092 ceph-mon[36049]: pgmap v286: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 542 KiB/s wr, 35 op/s 2024-05-27T08:47:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:31 smithi179 ceph-mon[36481]: pgmap v286: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 542 KiB/s wr, 35 op/s 2024-05-27T08:47:32.131 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:31 smithi077 ceph-mon[32007]: pgmap v286: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 542 KiB/s wr, 35 op/s 2024-05-27T08:47:32.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 310305 3.65 MB/sec execute 326 sec latency 877.130 ms 2024-05-27T08:47:33.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 310305 3.64 MB/sec execute 327 sec latency 1877.212 ms 2024-05-27T08:47:33.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:33 smithi092 ceph-mon[36049]: pgmap v287: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-05-27T08:47:34.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:33 smithi179 ceph-mon[36481]: pgmap v287: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-05-27T08:47:34.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:33 smithi077 ceph-mon[32007]: pgmap v287: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-05-27T08:47:34.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 310305 3.63 MB/sec execute 328 sec latency 2877.348 ms 2024-05-27T08:47:35.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 310305 3.62 MB/sec execute 329 sec latency 3877.431 ms 2024-05-27T08:47:35.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:35 smithi092 ceph-mon[36049]: pgmap v288: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 855 KiB/s wr, 75 op/s 2024-05-27T08:47:36.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:35 smithi179 ceph-mon[36481]: pgmap v288: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 855 KiB/s wr, 75 op/s 2024-05-27T08:47:36.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:35 smithi077 ceph-mon[32007]: pgmap v288: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 855 KiB/s wr, 75 op/s 2024-05-27T08:47:36.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 312379 3.65 MB/sec execute 330 sec latency 3942.671 ms 2024-05-27T08:47:37.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 313565 3.65 MB/sec execute 331 sec latency 17.730 ms 2024-05-27T08:47:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:37 smithi092 ceph-mon[36049]: pgmap v289: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 855 KiB/s wr, 75 op/s 2024-05-27T08:47:38.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:37 smithi179 ceph-mon[36481]: pgmap v289: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 855 KiB/s wr, 75 op/s 2024-05-27T08:47:38.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:37 smithi077 ceph-mon[32007]: pgmap v289: 97 pgs: 97 active+clean; 217 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 855 KiB/s wr, 75 op/s 2024-05-27T08:47:38.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 313847 3.64 MB/sec execute 332 sec latency 713.248 ms 2024-05-27T08:47:39.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 313847 3.63 MB/sec execute 333 sec latency 1713.370 ms 2024-05-27T08:47:39.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:39 smithi092 ceph-mon[36049]: pgmap v290: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-05-27T08:47:40.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:39 smithi179 ceph-mon[36481]: pgmap v290: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-05-27T08:47:40.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:39 smithi077 ceph-mon[32007]: pgmap v290: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-05-27T08:47:40.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 313847 3.62 MB/sec execute 334 sec latency 2713.468 ms 2024-05-27T08:47:40.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:40 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:47:41.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:40 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:47:41.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:40 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:47:41.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 315779 3.65 MB/sec execute 335 sec latency 2901.387 ms 2024-05-27T08:47:41.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:41 smithi092 ceph-mon[36049]: pgmap v291: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-05-27T08:47:42.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:41 smithi179 ceph-mon[36481]: pgmap v291: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-05-27T08:47:42.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:41 smithi077 ceph-mon[32007]: pgmap v291: 97 pgs: 97 active+clean; 220 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-05-27T08:47:42.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 316943 3.65 MB/sec execute 336 sec latency 13.630 ms 2024-05-27T08:47:43.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 317396 3.65 MB/sec execute 337 sec latency 751.483 ms 2024-05-27T08:47:43.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:43 smithi092 ceph-mon[36049]: pgmap v292: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-27T08:47:44.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:43 smithi179 ceph-mon[36481]: pgmap v292: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-27T08:47:44.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:43 smithi077 ceph-mon[32007]: pgmap v292: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 149 op/s 2024-05-27T08:47:44.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 317396 3.63 MB/sec execute 338 sec latency 1751.550 ms 2024-05-27T08:47:45.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 317396 3.62 MB/sec execute 339 sec latency 2751.641 ms 2024-05-27T08:47:45.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:45 smithi092 ceph-mon[36049]: pgmap v293: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 842 KiB/s wr, 90 op/s 2024-05-27T08:47:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:45 smithi179 ceph-mon[36481]: pgmap v293: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 842 KiB/s wr, 90 op/s 2024-05-27T08:47:46.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:45 smithi077 ceph-mon[32007]: pgmap v293: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 842 KiB/s wr, 90 op/s 2024-05-27T08:47:46.615 INFO:tasks.workunit.client.0.smithi077.stdout: 1 319320 3.65 MB/sec execute 340 sec latency 2938.554 ms 2024-05-27T08:47:47.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 320430 3.65 MB/sec execute 341 sec latency 16.836 ms 2024-05-27T08:47:47.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:47 smithi092 ceph-mon[36049]: pgmap v294: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 842 KiB/s wr, 90 op/s 2024-05-27T08:47:48.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:47 smithi179 ceph-mon[36481]: pgmap v294: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 842 KiB/s wr, 90 op/s 2024-05-27T08:47:48.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:47 smithi077 ceph-mon[32007]: pgmap v294: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 842 KiB/s wr, 90 op/s 2024-05-27T08:47:48.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 320938 3.65 MB/sec execute 342 sec latency 723.061 ms 2024-05-27T08:47:49.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 320938 3.64 MB/sec execute 343 sec latency 1723.181 ms 2024-05-27T08:47:49.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:49 smithi092 ceph-mon[36049]: pgmap v295: 97 pgs: 97 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-27T08:47:50.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:49 smithi179 ceph-mon[36481]: pgmap v295: 97 pgs: 97 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-27T08:47:50.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:49 smithi077 ceph-mon[32007]: pgmap v295: 97 pgs: 97 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-05-27T08:47:50.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 320938 3.63 MB/sec execute 344 sec latency 2723.241 ms 2024-05-27T08:47:51.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 323020 3.66 MB/sec execute 345 sec latency 2787.701 ms 2024-05-27T08:47:51.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:51 smithi092 ceph-mon[36049]: pgmap v296: 97 pgs: 97 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 669 KiB/s wr, 91 op/s 2024-05-27T08:47:52.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:51 smithi179 ceph-mon[36481]: pgmap v296: 97 pgs: 97 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 669 KiB/s wr, 91 op/s 2024-05-27T08:47:52.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:51 smithi077 ceph-mon[32007]: pgmap v296: 97 pgs: 97 active+clean; 227 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 669 KiB/s wr, 91 op/s 2024-05-27T08:47:52.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 324248 3.66 MB/sec execute 346 sec latency 17.079 ms 2024-05-27T08:47:53.617 INFO:tasks.workunit.client.0.smithi077.stdout: 1 324487 3.65 MB/sec execute 347 sec latency 759.194 ms 2024-05-27T08:47:53.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:53 smithi092 ceph-mon[36049]: pgmap v297: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-05-27T08:47:54.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:53 smithi179 ceph-mon[36481]: pgmap v297: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-05-27T08:47:54.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:53 smithi077 ceph-mon[32007]: pgmap v297: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-05-27T08:47:54.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 324487 3.64 MB/sec execute 348 sec latency 1759.261 ms 2024-05-27T08:47:55.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 324487 3.63 MB/sec execute 349 sec latency 2759.352 ms 2024-05-27T08:47:55.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:55 smithi092 ceph-mon[36049]: pgmap v298: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 758 KiB/s wr, 100 op/s 2024-05-27T08:47:56.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:55 smithi179 ceph-mon[36481]: pgmap v298: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 758 KiB/s wr, 100 op/s 2024-05-27T08:47:56.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:55 smithi077 ceph-mon[32007]: pgmap v298: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 758 KiB/s wr, 100 op/s 2024-05-27T08:47:56.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 326410 3.65 MB/sec execute 350 sec latency 2945.338 ms 2024-05-27T08:47:57.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 327589 3.65 MB/sec execute 351 sec latency 19.576 ms 2024-05-27T08:47:57.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:57 smithi092 ceph-mon[36049]: pgmap v299: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 758 KiB/s wr, 100 op/s 2024-05-27T08:47:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:57 smithi179 ceph-mon[36481]: pgmap v299: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 758 KiB/s wr, 100 op/s 2024-05-27T08:47:58.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:57 smithi077 ceph-mon[32007]: pgmap v299: 97 pgs: 97 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 758 KiB/s wr, 100 op/s 2024-05-27T08:47:58.616 INFO:tasks.workunit.client.0.smithi077.stdout: 1 328029 3.65 MB/sec execute 352 sec latency 605.933 ms 2024-05-27T08:47:59.617 INFO:tasks.workunit.client.0.smithi077.stdout: 1 328029 3.64 MB/sec execute 353 sec latency 1606.076 ms 2024-05-27T08:47:59.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:47:59 smithi092 ceph-mon[36049]: pgmap v300: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-05-27T08:48:00.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:47:59 smithi179 ceph-mon[36481]: pgmap v300: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-05-27T08:48:00.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:47:59 smithi077 ceph-mon[32007]: pgmap v300: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-05-27T08:48:00.617 INFO:tasks.workunit.client.0.smithi077.stdout: 1 328029 3.63 MB/sec execute 354 sec latency 2606.162 ms 2024-05-27T08:48:01.617 INFO:tasks.workunit.client.0.smithi077.stdout: 1 329215 3.64 MB/sec execute 355 sec latency 2793.199 ms 2024-05-27T08:48:01.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:01 smithi092 ceph-mon[36049]: pgmap v301: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 722 KiB/s wr, 96 op/s 2024-05-27T08:48:02.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:01 smithi179 ceph-mon[36481]: pgmap v301: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 722 KiB/s wr, 96 op/s 2024-05-27T08:48:02.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:01 smithi077 ceph-mon[32007]: pgmap v301: 97 pgs: 97 active+clean; 234 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 722 KiB/s wr, 96 op/s 2024-05-27T08:48:02.617 INFO:tasks.workunit.client.0.smithi077.stdout: 1 330505 3.65 MB/sec execute 356 sec latency 17.114 ms 2024-05-27T08:48:03.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:48:03.210 INFO:tasks.workunit.client.0.smithi077.stdout:[331410] write failed on handle 10469 (Resource temporarily unavailable) 2024-05-27T08:48:03.210 INFO:tasks.workunit.client.0.smithi077.stdout:Child failed with status 1 2024-05-27T08:48:03.210 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-05-27T08:48:03.210 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-27T08:48:03.488 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi077 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a9cb3a581a309a99b72997ae5ddb88084f5484c9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-27T08:48:03.753 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=adc9ceb56d364867a7c3b5c4b5ae07a8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_a9cb3a581a309a99b72997ae5ddb88084f5484c9/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e7ae7580c417d6ab27fda98fecadca5d9f85d63e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi077 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a9cb3a581a309a99b72997ae5ddb88084f5484c9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-05-27T08:48:03.756 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-27T08:48:03.765 INFO:tasks.quiescer:joining Quiescers 2024-05-27T08:48:03.765 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-27T08:48:03.766 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:48:03.793 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:03 smithi077 ceph-mon[32007]: pgmap v302: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-05-27T08:48:03.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:03 smithi092 ceph-mon[36049]: pgmap v302: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-05-27T08:48:04.013 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:48:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:03 smithi179 ceph-mon[36481]: pgmap v302: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-05-27T08:48:04.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:04 smithi092 ceph-mon[36049]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T08:48:05.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:04 smithi179 ceph-mon[36481]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T08:48:05.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:04 smithi077 ceph-mon[32007]: osdmap e93: 12 total, 12 up, 12 in 2024-05-27T08:48:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 -- 172.21.15.77:0/3364293986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 msgr2=0x7fa24806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 --2- 172.21.15.77:0/3364293986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 0x7fa24806bf30 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa234009990 tx=0x7fa234009ca0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 -- 172.21.15.77:0/3364293986 shutdown_connections 2024-05-27T08:48:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 --2- 172.21.15.77:0/3364293986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa24806c470 0x7fa2481103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 --2- 172.21.15.77:0/3364293986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2481095a0 0x7fa248109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.722 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 --2- 172.21.15.77:0/3364293986 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 0x7fa24806bf30 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 -- 172.21.15.77:0/3364293986 >> 172.21.15.77:0/3364293986 conn(0x7fa2480793b0 msgr2=0x7fa2480797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 -- 172.21.15.77:0/3364293986 shutdown_connections 2024-05-27T08:48:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.719+0000 7fa24ee97700 1 -- 172.21.15.77:0/3364293986 wait complete. 2024-05-27T08:48:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24ee97700 1 Processor -- start 2024-05-27T08:48:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24ee97700 1 -- start start 2024-05-27T08:48:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24ee97700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 0x7fa248102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24ee97700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa24806c470 0x7fa248102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:05.723 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24ee97700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2481095a0 0x7fa248102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24ee97700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2481137b0 con 0x7fa24806bab0 2024-05-27T08:48:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24ee97700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa248113630 con 0x7fa2481095a0 2024-05-27T08:48:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24ee97700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa248113930 con 0x7fa24806c470 2024-05-27T08:48:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24cc33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 0x7fa248102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24cc33700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 0x7fa248102500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44406/0 (socket says 172.21.15.77:44406) 2024-05-27T08:48:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24cc33700 1 -- 172.21.15.77:0/73496461 learned_addr learned my addr 172.21.15.77:0/73496461 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa247fff700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa24806c470 0x7fa248102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:05.724 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.720+0000 7fa24d434700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2481095a0 0x7fa248102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa24cc33700 1 -- 172.21.15.77:0/73496461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa24806c470 msgr2=0x7fa248102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa24cc33700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa24806c470 0x7fa248102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa24cc33700 1 -- 172.21.15.77:0/73496461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2481095a0 msgr2=0x7fa248102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa24cc33700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2481095a0 0x7fa248102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa24cc33700 1 -- 172.21.15.77:0/73496461 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2481a8290 con 0x7fa24806bab0 2024-05-27T08:48:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa24cc33700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 0x7fa248102500 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00b530 tx=0x7fa23c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa245ffb700 1 -- 172.21.15.77:0/73496461 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c015020 con 0x7fa24806bab0 2024-05-27T08:48:05.725 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa245ffb700 1 -- 172.21.15.77:0/73496461 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23c011ce0 con 0x7fa24806bab0 2024-05-27T08:48:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2481a8580 con 0x7fa24806bab0 2024-05-27T08:48:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa245ffb700 1 -- 172.21.15.77:0/73496461 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c010dd0 con 0x7fa24806bab0 2024-05-27T08:48:05.726 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.721+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2481a8a90 con 0x7fa24806bab0 2024-05-27T08:48:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.722+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24810ba80 con 0x7fa24806bab0 2024-05-27T08:48:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.724+0000 7fa245ffb700 1 -- 172.21.15.77:0/73496461 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa23c018020 con 0x7fa24806bab0 2024-05-27T08:48:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.725+0000 7fa245ffb700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa2380b29f0 0x7fa2380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.725+0000 7fa247fff700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa2380b29f0 0x7fa2380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:05.727 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.725+0000 7fa245ffb700 1 -- 172.21.15.77:0/73496461 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7fa23c0e2100 con 0x7fa24806bab0 2024-05-27T08:48:05.728 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.725+0000 7fa247fff700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa2380b29f0 0x7fa2380b4eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2340098e0 tx=0x7fa234005960 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:05.731 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.731+0000 7fa245ffb700 1 -- 172.21.15.77:0/73496461 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa23c01f020 con 0x7fa24806bab0 2024-05-27T08:48:05.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:05 smithi077 ceph-mon[32007]: pgmap v304: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 906 KiB/s wr, 151 op/s 2024-05-27T08:48:05.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:05 smithi092 ceph-mon[36049]: pgmap v304: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 906 KiB/s wr, 151 op/s 2024-05-27T08:48:05.987 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.987+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa24806a230 con 0x7fa24806bab0 2024-05-27T08:48:05.988 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.988+0000 7fa245ffb700 1 -- 172.21.15.77:0/73496461 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11927 (secure 0 0 0) 0x7fa23c0a9c50 con 0x7fa24806bab0 2024-05-27T08:48:05.988 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:48:05.989 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:47:38.676764+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24418,24434,24410,24412,24433]},"id":1}]} 2024-05-27T08:48:05.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa2380b29f0 msgr2=0x7fa2380b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:05.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa2380b29f0 0x7fa2380b4eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2340098e0 tx=0x7fa234005960 comp rx=0 tx=0).stop 2024-05-27T08:48:05.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 msgr2=0x7fa248102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:05.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 0x7fa248102500 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00b530 tx=0x7fa23c00b840 comp rx=0 tx=0).stop 2024-05-27T08:48:05.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 shutdown_connections 2024-05-27T08:48:05.991 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa2380b29f0 0x7fa2380b4eb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2481095a0 0x7fa248102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa24806bab0 0x7fa248102500 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 --2- 172.21.15.77:0/73496461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa24806c470 0x7fa248102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:05.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.990+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 >> 172.21.15.77:0/73496461 conn(0x7fa2480793b0 msgr2=0x7fa248101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:05.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.991+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 shutdown_connections 2024-05-27T08:48:05.992 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:05.991+0000 7fa24ee97700 1 -- 172.21.15.77:0/73496461 wait complete. 2024-05-27T08:48:05.993 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 39 2024-05-27T08:48:06.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:05 smithi179 ceph-mon[36481]: pgmap v304: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 906 KiB/s wr, 151 op/s 2024-05-27T08:48:06.655 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --cancel --all' 2024-05-27T08:48:06.655 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24434 quiesce db --cancel --all 2024-05-27T08:48:06.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:06 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/73496461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:48:06.949 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:48:06.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:06 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/73496461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:48:07.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:06 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/73496461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:48:07.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:07 smithi092 ceph-mon[36049]: pgmap v305: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 906 KiB/s wr, 151 op/s 2024-05-27T08:48:07.998 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:07 smithi077 ceph-mon[32007]: pgmap v305: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 906 KiB/s wr, 151 op/s 2024-05-27T08:48:08.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:07 smithi179 ceph-mon[36481]: pgmap v305: 97 pgs: 97 active+clean; 237 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 906 KiB/s wr, 151 op/s 2024-05-27T08:48:08.577 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 -- 172.21.15.77:0/1763392564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 msgr2=0x7f30781028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 --2- 172.21.15.77:0/1763392564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 0x7f30781028c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3060009990 tx=0x7f3060009ca0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 -- 172.21.15.77:0/1763392564 shutdown_connections 2024-05-27T08:48:08.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 --2- 172.21.15.77:0/1763392564 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078102e00 0x7f307810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 --2- 172.21.15.77:0/1763392564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3078108520 0x7f3078108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.578 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 --2- 172.21.15.77:0/1763392564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 0x7f30781028c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 -- 172.21.15.77:0/1763392564 >> 172.21.15.77:0/1763392564 conn(0x7f30780793f0 msgr2=0x7f3078079800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 -- 172.21.15.77:0/1763392564 shutdown_connections 2024-05-27T08:48:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.575+0000 7f307e0d4700 1 -- 172.21.15.77:0/1763392564 wait complete. 2024-05-27T08:48:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.576+0000 7f307e0d4700 1 Processor -- start 2024-05-27T08:48:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.576+0000 7f307e0d4700 1 -- start start 2024-05-27T08:48:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.576+0000 7f307e0d4700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 0x7f307819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.576+0000 7f307e0d4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3078102e00 0x7f307819b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.579 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.576+0000 7f307e0d4700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078108520 0x7f30781a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.576+0000 7f307e0d4700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3078112650 con 0x7f3078102440 2024-05-27T08:48:08.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.576+0000 7f307e0d4700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30781124d0 con 0x7f3078108520 2024-05-27T08:48:08.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.576+0000 7f307e0d4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f30781127d0 con 0x7f3078102e00 2024-05-27T08:48:08.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 0x7f307819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 0x7f307819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44420/0 (socket says 172.21.15.77:44420) 2024-05-27T08:48:08.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 -- 172.21.15.77:0/723058794 learned_addr learned my addr 172.21.15.77:0/723058794 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:08.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f3077fff700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078108520 0x7f30781a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.580 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f306effd700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3078102e00 0x7f307819b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 -- 172.21.15.77:0/723058794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3078102e00 msgr2=0x7f307819b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3078102e00 0x7f307819b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 -- 172.21.15.77:0/723058794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078108520 msgr2=0x7f30781a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078108520 0x7f30781a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 -- 172.21.15.77:0/723058794 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30781a60d0 con 0x7f3078102440 2024-05-27T08:48:08.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f3077fff700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078108520 0x7f30781a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:48:08.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30777fe700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 0x7f307819ab10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f306800b530 tx=0x7f306800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:08.581 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30757fa700 1 -- 172.21.15.77:0/723058794 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068015020 con 0x7f3078102440 2024-05-27T08:48:08.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30757fa700 1 -- 172.21.15.77:0/723058794 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3068011ce0 con 0x7f3078102440 2024-05-27T08:48:08.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f307e0d4700 1 -- 172.21.15.77:0/723058794 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30781a6300 con 0x7f3078102440 2024-05-27T08:48:08.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f307e0d4700 1 -- 172.21.15.77:0/723058794 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30781a67c0 con 0x7f3078102440 2024-05-27T08:48:08.582 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.577+0000 7f30757fa700 1 -- 172.21.15.77:0/723058794 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068010d20 con 0x7f3078102440 2024-05-27T08:48:08.587 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.580+0000 7f30757fa700 1 -- 172.21.15.77:0/723058794 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3068018020 con 0x7f3078102440 2024-05-27T08:48:08.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f30757fa700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f30640b29f0 0x7f30640b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f306effd700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f30640b29f0 0x7f30640b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f30757fa700 1 -- 172.21.15.77:0/723058794 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f30680e1de0 con 0x7f3078102440 2024-05-27T08:48:08.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f308896ab80 1 Processor -- start 2024-05-27T08:48:08.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f308896ab80 1 -- start start 2024-05-27T08:48:08.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f306effd700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f30640b29f0 0x7f30640b4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3060006010 tx=0x7f3060005c40 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:08.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f308896ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7da5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.588 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f308896ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7d9fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f308896ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7dac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f308896ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564fa7cfed00 con 0x564fa7da0210 2024-05-27T08:48:08.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f308896ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564fa7cee8a0 con 0x564fa7da5220 2024-05-27T08:48:08.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f308896ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564fa7cf28e0 con 0x564fa7d9f850 2024-05-27T08:48:08.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.581+0000 7f3077fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7dac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f3077fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7dac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44424/0 (socket says 172.21.15.77:44424) 2024-05-27T08:48:08.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f3077fff700 1 -- 172.21.15.77:0/3064788815 learned_addr learned my addr 172.21.15.77:0/3064788815 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:08.589 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f30777fe700 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7da5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f306effd700 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7d9fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f3077fff700 1 -- 172.21.15.77:0/3064788815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 msgr2=0x564fa7d9fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:48:08.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f3077fff700 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7d9fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f3077fff700 1 -- 172.21.15.77:0/3064788815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 msgr2=0x564fa7da5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:48:08.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f3077fff700 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7da5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f3077fff700 1 -- 172.21.15.77:0/3064788815 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa7c3e940 con 0x564fa7da0210 2024-05-27T08:48:08.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f3077fff700 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7dac050 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f30700087c0 tx=0x7f3070008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1b139b70dcfd60 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:08.590 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f306dffb700 1 -- 172.21.15.77:0/3064788815 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070003c00 con 0x564fa7da0210 2024-05-27T08:48:08.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f306dffb700 1 -- 172.21.15.77:0/3064788815 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3070003da0 con 0x564fa7da0210 2024-05-27T08:48:08.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.582+0000 7f306dffb700 1 -- 172.21.15.77:0/3064788815 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070012970 con 0x564fa7da0210 2024-05-27T08:48:08.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 -- 172.21.15.77:0/3064788815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 msgr2=0x564fa7dac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7dac050 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f30700087c0 tx=0x7f3070008ad0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 -- 172.21.15.77:0/3064788815 shutdown_connections 2024-05-27T08:48:08.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7dac050 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7da5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.591 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 --2- 172.21.15.77:0/3064788815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7d9fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 -- 172.21.15.77:0/3064788815 >> 172.21.15.77:0/3064788815 conn(0x564fa7ccd0f0 msgr2=0x564fa7d9a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:08.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 -- 172.21.15.77:0/3064788815 shutdown_connections 2024-05-27T08:48:08.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 -- 172.21.15.77:0/3064788815 wait complete. 2024-05-27T08:48:08.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.583+0000 7f308896ab80 1 -- start start 2024-05-27T08:48:08.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f308896ab80 1 Processor -- start 2024-05-27T08:48:08.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f308896ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7dabd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f308896ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da0210 0x564fa7da9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.592 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f308896ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da5220 0x564fa7da9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f308896ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564fa7bf5010 con 0x564fa7da5220 2024-05-27T08:48:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f308896ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x564fa7c90830 con 0x564fa7da0210 2024-05-27T08:48:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f308896ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564fa7dac650 con 0x564fa7d9f850 2024-05-27T08:48:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f308896ab80 10 client.0 _ll_register_callbacks cb 0x564fa7c89990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:48:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f3077fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da5220 0x564fa7da9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f3077fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da5220 0x564fa7da9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44438/0 (socket says 172.21.15.77:44438) 2024-05-27T08:48:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f3077fff700 1 -- 172.21.15.77:0/2137579026 learned_addr learned my addr 172.21.15.77:0/2137579026 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:08.593 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f306effd700 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da0210 0x564fa7da9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f30777fe700 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7dabd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f3077fff700 1 -- 172.21.15.77:0/2137579026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 msgr2=0x564fa7dabd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:48:08.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f3077fff700 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7dabd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f3077fff700 1 -- 172.21.15.77:0/2137579026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da0210 msgr2=0x564fa7da9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:48:08.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f3077fff700 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da0210 0x564fa7da9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.584+0000 7f3077fff700 1 -- 172.21.15.77:0/2137579026 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa7c8d700 con 0x564fa7da5220 2024-05-27T08:48:08.594 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f3077fff700 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da5220 0x564fa7da9540 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3070007b30 tx=0x7f3070004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7440fa62dbb72b server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:08.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f3056ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:48:08.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f3056ffd700 1 -- 172.21.15.77:0/2137579026 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070004ce0 con 0x564fa7da5220 2024-05-27T08:48:08.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f3056ffd700 1 -- 172.21.15.77:0/2137579026 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f3070004360 con 0x564fa7da5220 2024-05-27T08:48:08.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 1 -- 172.21.15.77:0/2137579026 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3070003da0 con 0x564fa7da5220 2024-05-27T08:48:08.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f3056ffd700 1 -- 172.21.15.77:0/2137579026 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f307002b050 con 0x564fa7da5220 2024-05-27T08:48:08.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 10 client.15789 fetch_fsmap learned FSMap version 39 2024-05-27T08:48:08.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 1 -- 172.21.15.77:0/2137579026 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564fa7db6d80 con 0x564fa7da5220 2024-05-27T08:48:08.595 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f3056ffd700 1 -- 172.21.15.77:0/2137579026 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3070003da0 con 0x564fa7da5220 2024-05-27T08:48:08.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f3056ffd700 1 -- 172.21.15.77:0/2137579026 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3070013070 con 0x564fa7da5220 2024-05-27T08:48:08.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 10 client.15789 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T08:48:08.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 10 client.15789 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:48:08.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 20 client.15789 populate_metadata read hostname 'smithi077' 2024-05-27T08:48:08.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564fa7db9300 0x564fa7dbb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 4 client.15789 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-27T08:48:08.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.585+0000 7f308896ab80 1 -- 172.21.15.77:0/2137579026 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fa7d6ba50 con 0x564fa7db9300 2024-05-27T08:48:08.596 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.586+0000 7f30777fe700 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564fa7db9300 0x564fa7dbb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.586+0000 7f3056ffd700 10 client.15789 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.586+0000 7f30777fe700 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564fa7db9300 0x564fa7dbb6f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab26c564d03a394c server_cookie=42e56c69bccc9e36 in_seq=0 out_seq=0 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f3056ffd700 1 -- 172.21.15.77:0/2137579026 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564fa7d6ba50 con 0x564fa7db9300 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f3056ffd700 10 client.15789 handle_command_reply: tid=0 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 client.15789 shutdown 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 -- 172.21.15.77:0/2137579026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da5220 msgr2=0x564fa7da9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da5220 0x564fa7da9540 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3070007b30 tx=0x7f3070004180 comp rx=0 tx=0).stop 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 -- 172.21.15.77:0/2137579026 shutdown_connections 2024-05-27T08:48:08.597 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564fa7db9300 0x564fa7dbb6f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da5220 0x564fa7da9540 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7dabd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 --2- 172.21.15.77:0/2137579026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da0210 0x564fa7da9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 -- 172.21.15.77:0/2137579026 >> 172.21.15.77:0/2137579026 conn(0x564fa7ccd0f0 msgr2=0x564fa7d94c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:08.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f3056ffd700 0 client.15789 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:48:08.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 -- 172.21.15.77:0/2137579026 shutdown_connections 2024-05-27T08:48:08.598 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 1 -- 172.21.15.77:0/2137579026 wait complete. 2024-05-27T08:48:08.599 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.587+0000 7f308896ab80 20 client.15789 trim_cache size 0 max 16384 2024-05-27T08:48:08.608 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.607+0000 7f308896ab80 1 Processor -- start 2024-05-27T08:48:08.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.607+0000 7f308896ab80 1 -- start start 2024-05-27T08:48:08.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f308896ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7d9f850 0x564fa7dab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f308896ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7da0210 0x564fa7d94380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f308896ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7d948c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f308896ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30781ae070 con 0x564fa7d9f850 2024-05-27T08:48:08.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f308896ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30781afbb0 con 0x564fa7da5220 2024-05-27T08:48:08.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f308896ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f30781ae740 con 0x564fa7da0210 2024-05-27T08:48:08.609 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7d9f850 0x564fa7dab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7d9f850 0x564fa7dab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44446/0 (socket says 172.21.15.77:44446) 2024-05-27T08:48:08.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 -- 172.21.15.77:0/695774997 learned_addr learned my addr 172.21.15.77:0/695774997 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:08.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 -- 172.21.15.77:0/695774997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7da0210 msgr2=0x564fa7d94380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:48:08.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 --2- 172.21.15.77:0/695774997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7da0210 0x564fa7d94380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 -- 172.21.15.77:0/695774997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 msgr2=0x564fa7d948c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:48:08.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 --2- 172.21.15.77:0/695774997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7d948c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.610 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 -- 172.21.15.77:0/695774997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa7da7590 con 0x564fa7d9f850 2024-05-27T08:48:08.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f30777fe700 1 --2- 172.21.15.77:0/695774997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7d9f850 0x564fa7dab840 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f30680df660 tx=0x7f306801c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e51dc2c47551d2d server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:08.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f306d7fa700 1 -- 172.21.15.77:0/695774997 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30680a5420 con 0x564fa7d9f850 2024-05-27T08:48:08.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f306d7fa700 1 -- 172.21.15.77:0/695774997 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30680ad160 con 0x564fa7d9f850 2024-05-27T08:48:08.611 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.608+0000 7f306d7fa700 1 -- 172.21.15.77:0/695774997 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30680b1dc0 con 0x564fa7d9f850 2024-05-27T08:48:08.614 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 -- 172.21.15.77:0/695774997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7d9f850 msgr2=0x564fa7dab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 --2- 172.21.15.77:0/695774997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7d9f850 0x564fa7dab840 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f30680df660 tx=0x7f306801c040 comp rx=0 tx=0).stop 2024-05-27T08:48:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 -- 172.21.15.77:0/695774997 shutdown_connections 2024-05-27T08:48:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 --2- 172.21.15.77:0/695774997 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7d948c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 --2- 172.21.15.77:0/695774997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7d9f850 0x564fa7dab840 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 --2- 172.21.15.77:0/695774997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7da0210 0x564fa7d94380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 -- 172.21.15.77:0/695774997 >> 172.21.15.77:0/695774997 conn(0x564fa7ccd0f0 msgr2=0x564fa7d9f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 -- 172.21.15.77:0/695774997 shutdown_connections 2024-05-27T08:48:08.615 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 -- 172.21.15.77:0/695774997 wait complete. 2024-05-27T08:48:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.609+0000 7f308896ab80 1 -- start start 2024-05-27T08:48:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f308896ab80 1 Processor -- start 2024-05-27T08:48:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f308896ab80 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7db1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f308896ab80 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7db2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f308896ab80 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7dae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f308896ab80 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30781af4e0 con 0x564fa7da0210 2024-05-27T08:48:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f308896ab80 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f30781aee10 con 0x564fa7da5220 2024-05-27T08:48:08.616 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f308896ab80 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f30781ac030 con 0x564fa7d9f850 2024-05-27T08:48:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f308896ab80 10 client.0 _ll_register_callbacks cb 0x564fa7c89990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-27T08:48:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7db2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7db2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:44450/0 (socket says 172.21.15.77:44450) 2024-05-27T08:48:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 -- 172.21.15.77:0/4210279327 learned_addr learned my addr 172.21.15.77:0/4210279327 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f30777fe700 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7db1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f3077fff700 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7dae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 -- 172.21.15.77:0/4210279327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 msgr2=0x564fa7db1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:48:08.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7db1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 -- 172.21.15.77:0/4210279327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 msgr2=0x564fa7dae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-27T08:48:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7dae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 -- 172.21.15.77:0/4210279327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa7daf0c0 con 0x564fa7da0210 2024-05-27T08:48:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f306effd700 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7db2000 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3060009990 tx=0x7f3060009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5406bc3bbc467a22 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.610+0000 7f3056ffd700 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-05-27T08:48:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f3056ffd700 1 -- 172.21.15.77:0/4210279327 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306000b9e0 con 0x564fa7da0210 2024-05-27T08:48:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f3056ffd700 1 -- 172.21.15.77:0/4210279327 <== mon.0 v2:172.21.15.77:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8586+0+0 (secure 0 0 0) 0x7f306000bb80 con 0x564fa7da0210 2024-05-27T08:48:08.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 1 -- 172.21.15.77:0/4210279327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30680ad160 con 0x564fa7da0210 2024-05-27T08:48:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f3056ffd700 1 -- 172.21.15.77:0/4210279327 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f306000bd80 con 0x564fa7da0210 2024-05-27T08:48:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 10 client.15795 fetch_fsmap learned FSMap version 39 2024-05-27T08:48:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 1 -- 172.21.15.77:0/4210279327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564fa7e247f0 con 0x564fa7da0210 2024-05-27T08:48:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f3056ffd700 1 -- 172.21.15.77:0/4210279327 <== mon.0 v2:172.21.15.77:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30680ad160 con 0x564fa7da0210 2024-05-27T08:48:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f3056ffd700 1 -- 172.21.15.77:0/4210279327 <== mon.0 v2:172.21.15.77:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f3060020040 con 0x564fa7da0210 2024-05-27T08:48:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 10 client.15795 fetch_fsmap finished waiting for FSMap version 39 2024-05-27T08:48:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 10 client.15795 resolve_mds: validated gid 24434 aka daemon mds.b 2024-05-27T08:48:08.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 20 client.15795 populate_metadata read hostname 'smithi077' 2024-05-27T08:48:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564fa7e24f80 0x564fa7e273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 4 client.15795 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-27T08:48:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.611+0000 7f308896ab80 1 -- 172.21.15.77:0/4210279327 --> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x564fa7d00c50 con 0x564fa7e24f80 2024-05-27T08:48:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f30777fe700 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564fa7e24f80 0x564fa7e273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f3056ffd700 10 client.15795 ms_handle_connect on v2:172.21.15.92:6838/709031605 2024-05-27T08:48:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f30777fe700 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564fa7e24f80 0x564fa7e273a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a825b36fe8225d server_cookie=5cb5b0c445fb0bff in_seq=0 out_seq=0 2024-05-27T08:48:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f3056ffd700 1 -- 172.21.15.77:0/4210279327 <== mds.0 v2:172.21.15.92:6838/709031605 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x564fa7d00c50 con 0x564fa7e24f80 2024-05-27T08:48:08.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f3056ffd700 10 client.15795 handle_command_reply: tid=0 2024-05-27T08:48:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f308896ab80 1 client.15795 shutdown 2024-05-27T08:48:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f308896ab80 1 -- 172.21.15.77:0/4210279327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 msgr2=0x564fa7db2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f308896ab80 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7db2000 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3060009990 tx=0x7f3060009310 comp rx=0 tx=0).stop 2024-05-27T08:48:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f308896ab80 1 -- 172.21.15.77:0/4210279327 shutdown_connections 2024-05-27T08:48:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.612+0000 7f308896ab80 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.92:6838/709031605,v1:172.21.15.92:6839/709031605] conn(0x564fa7e24f80 0x564fa7e273a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f308896ab80 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x564fa7da5220 0x564fa7dae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f308896ab80 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fa7d9f850 0x564fa7db1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f308896ab80 1 --2- 172.21.15.77:0/4210279327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa7da0210 0x564fa7db2000 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f308896ab80 1 -- 172.21.15.77:0/4210279327 >> 172.21.15.77:0/4210279327 conn(0x564fa7ccd0f0 msgr2=0x564fa7da2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:08.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f3056ffd700 0 client.15795 ms_handle_reset on v2:172.21.15.92:6838/709031605 2024-05-27T08:48:08.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f308896ab80 1 -- 172.21.15.77:0/4210279327 shutdown_connections 2024-05-27T08:48:08.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f308896ab80 1 -- 172.21.15.77:0/4210279327 wait complete. 2024-05-27T08:48:08.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f308896ab80 20 client.15795 trim_cache size 0 max 16384 2024-05-27T08:48:08.622 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:48:08.623 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 39, 2024-05-27T08:48:08.623 INFO:teuthology.orchestra.run.smithi077.stdout: "leader": 24434, 2024-05-27T08:48:08.623 INFO:teuthology.orchestra.run.smithi077.stdout: "set_version": 28, 2024-05-27T08:48:08.623 INFO:teuthology.orchestra.run.smithi077.stdout: "sets": {} 2024-05-27T08:48:08.623 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:48:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 -- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f30640b29f0 msgr2=0x7f30640b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f30640b29f0 0x7f30640b4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3060006010 tx=0x7f3060005c40 comp rx=0 tx=0).stop 2024-05-27T08:48:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 -- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 msgr2=0x7f307819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 0x7f307819ab10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f306800b530 tx=0x7f306800b8f0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 -- 172.21.15.77:0/723058794 shutdown_connections 2024-05-27T08:48:08.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f30640b29f0 0x7f30640b4eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3078108520 0x7f30781a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3078102440 0x7f307819ab10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 --2- 172.21.15.77:0/723058794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3078102e00 0x7f307819b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 -- 172.21.15.77:0/723058794 >> 172.21.15.77:0/723058794 conn(0x7f30780793f0 msgr2=0x7f30781005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 -- 172.21.15.77:0/723058794 shutdown_connections 2024-05-27T08:48:08.625 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:08.613+0000 7f307e0d4700 1 -- 172.21.15.77:0/723058794 wait complete. 2024-05-27T08:48:09.417 INFO:tasks.quiescer:done joining Quiescers 2024-05-27T08:48:09.418 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-27T08:48:09.555 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-27T08:48:09.556 INFO:teuthology.task.exec:Executing custom commands... 2024-05-27T08:48:09.556 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:48:09.556 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-27T08:48:09.587 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-27T08:48:09.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:09 smithi077 ceph-mon[32007]: pgmap v306: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-27T08:48:09.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:09 smithi092 ceph-mon[36049]: pgmap v306: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-27T08:48:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:09 smithi179 ceph-mon[36481]: pgmap v306: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-27T08:48:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:11 smithi092 ceph-mon[36049]: pgmap v307: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-27T08:48:11.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:11 smithi092 ceph-mon[36049]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T08:48:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:11 smithi179 ceph-mon[36481]: pgmap v307: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-27T08:48:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:11 smithi179 ceph-mon[36481]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T08:48:12.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:11 smithi077 ceph-mon[32007]: pgmap v307: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-05-27T08:48:12.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:11 smithi077 ceph-mon[32007]: osdmap e94: 12 total, 12 up, 12 in 2024-05-27T08:48:13.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:13 smithi092 ceph-mon[36049]: pgmap v309: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-05-27T08:48:14.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:13 smithi179 ceph-mon[36481]: pgmap v309: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-05-27T08:48:14.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:13 smithi077 ceph-mon[32007]: pgmap v309: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 76 op/s 2024-05-27T08:48:15.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:15 smithi092 ceph-mon[36049]: pgmap v310: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 960 KiB/s wr, 66 op/s 2024-05-27T08:48:16.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:15 smithi179 ceph-mon[36481]: pgmap v310: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 960 KiB/s wr, 66 op/s 2024-05-27T08:48:16.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:15 smithi077 ceph-mon[32007]: pgmap v310: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 960 KiB/s wr, 66 op/s 2024-05-27T08:48:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:17 smithi092 ceph-mon[36049]: pgmap v311: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 960 KiB/s wr, 66 op/s 2024-05-27T08:48:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:17 smithi179 ceph-mon[36481]: pgmap v311: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 960 KiB/s wr, 66 op/s 2024-05-27T08:48:18.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:17 smithi077 ceph-mon[32007]: pgmap v311: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 960 KiB/s wr, 66 op/s 2024-05-27T08:48:18.660 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-27T08:48:18.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 -- 172.21.15.77:0/3744629911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 msgr2=0x7efeb410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:18.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/3744629911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 0x7efeb410f930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efeac009cd0 tx=0x7efeac02a8e0 comp rx=0 tx=0).stop 2024-05-27T08:48:18.782 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 -- 172.21.15.77:0/3744629911 shutdown_connections 2024-05-27T08:48:18.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/3744629911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 0x7efeb410f930 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:18.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/3744629911 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeb4102a60 0x7efeb4102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:18.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/3744629911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4108a70 0x7efeb4108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:18.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 -- 172.21.15.77:0/3744629911 >> 172.21.15.77:0/3744629911 conn(0x7efeb4079ee0 msgr2=0x7efeb407a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:18.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 -- 172.21.15.77:0/3744629911 shutdown_connections 2024-05-27T08:48:18.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.780+0000 7efeb9e1d640 1 -- 172.21.15.77:0/3744629911 wait complete. 2024-05-27T08:48:18.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb9e1d640 1 Processor -- start 2024-05-27T08:48:18.783 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb9e1d640 1 -- start start 2024-05-27T08:48:18.785 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb9e1d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4102a60 0x7efeb41991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:18.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb9e1d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 0x7efeb41996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:18.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb9e1d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeb4108a70 0x7efeb419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:18.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb9e1d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4112d30 con 0x7efeb4103420 2024-05-27T08:48:18.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb9e1d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4112bb0 con 0x7efeb4108a70 2024-05-27T08:48:18.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb9e1d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4112eb0 con 0x7efeb4102a60 2024-05-27T08:48:18.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 0x7efeb41996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:18.786 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 0x7efeb41996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48402/0 (socket says 172.21.15.77:48402) 2024-05-27T08:48:18.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb2ffd640 1 -- 172.21.15.77:0/2438462601 learned_addr learned my addr 172.21.15.77:0/2438462601 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:18.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.781+0000 7efeb37fe640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4102a60 0x7efeb41991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:18.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb3fff640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeb4108a70 0x7efeb419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:18.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb2ffd640 1 -- 172.21.15.77:0/2438462601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4102a60 msgr2=0x7efeb41991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:18.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb2ffd640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4102a60 0x7efeb41991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:18.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb2ffd640 1 -- 172.21.15.77:0/2438462601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeb4108a70 msgr2=0x7efeb419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:18.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb2ffd640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeb4108a70 0x7efeb419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:18.787 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb2ffd640 1 -- 172.21.15.77:0/2438462601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb419e150 con 0x7efeb4103420 2024-05-27T08:48:18.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb3fff640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeb4108a70 0x7efeb419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:48:18.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb2ffd640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 0x7efeb41996f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efea800d950 tx=0x7efea800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:18.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb0ff9640 1 -- 172.21.15.77:0/2438462601 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8017030 con 0x7efeb4103420 2024-05-27T08:48:18.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb419e440 con 0x7efeb4103420 2024-05-27T08:48:18.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb0ff9640 1 -- 172.21.15.77:0/2438462601 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea8004020 con 0x7efeb4103420 2024-05-27T08:48:18.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb0ff9640 1 -- 172.21.15.77:0/2438462601 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80047a0 con 0x7efeb4103420 2024-05-27T08:48:18.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.782+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb41a5cd0 con 0x7efeb4103420 2024-05-27T08:48:18.788 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.783+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb40507e0 con 0x7efeb4103420 2024-05-27T08:48:18.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.785+0000 7efeb0ff9640 1 -- 172.21.15.77:0/2438462601 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efea8009a30 con 0x7efeb4103420 2024-05-27T08:48:18.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.786+0000 7efeb0ff9640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7efe9c0b2a10 0x7efe9c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:18.789 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.786+0000 7efeb37fe640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7efe9c0b2a10 0x7efe9c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:18.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.786+0000 7efeb0ff9640 1 -- 172.21.15.77:0/2438462601 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7efea80e26b0 con 0x7efeb4103420 2024-05-27T08:48:18.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.787+0000 7efeb37fe640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7efe9c0b2a10 0x7efe9c0b4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efea40058c0 tx=0x7efea400a250 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:18.790 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.790+0000 7efeb0ff9640 1 -- 172.21.15.77:0/2438462601 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efea80aa080 con 0x7efeb4103420 2024-05-27T08:48:18.996 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:18.995+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7efeb4062e90 con 0x7efe9c0b2a10 2024-05-27T08:48:19.004 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.003+0000 7efeb0ff9640 1 -- 172.21.15.77:0/2438462601 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7efeb4062e90 con 0x7efe9c0b2a10 2024-05-27T08:48:19.005 INFO:teuthology.orchestra.run.smithi077.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-27T00:00:00", "created": "2024-05-27T08:38:17", "first": "2024-05-27T08:39:00", "last": "2024-05-27T08:48:00", "last_pruned": "2024-05-27T08:48:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-27T08:48:19.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.005+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7efe9c0b2a10 msgr2=0x7efe9c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.005+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7efe9c0b2a10 0x7efe9c0b4ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efea40058c0 tx=0x7efea400a250 comp rx=0 tx=0).stop 2024-05-27T08:48:19.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.005+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 msgr2=0x7efeb41996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.005+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 0x7efeb41996f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efea800d950 tx=0x7efea800de10 comp rx=0 tx=0).stop 2024-05-27T08:48:19.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.006+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 shutdown_connections 2024-05-27T08:48:19.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.006+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7efe9c0b2a10 0x7efe9c0b4ed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.006+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efeb4108a70 0x7efeb419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.006+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efeb4103420 0x7efeb41996f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.006+0000 7efeb9e1d640 1 --2- 172.21.15.77:0/2438462601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efeb4102a60 0x7efeb41991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.006+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 >> 172.21.15.77:0/2438462601 conn(0x7efeb4079ee0 msgr2=0x7efeb4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:19.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.006+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 shutdown_connections 2024-05-27T08:48:19.009 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.006+0000 7efeb9e1d640 1 -- 172.21.15.77:0/2438462601 wait complete. 2024-05-27T08:48:19.018 DEBUG:teuthology.orchestra.run.smithi077:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-27T08:48:19.125 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 -- 172.21.15.77:0/675177107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974107940 msgr2=0x7f9974107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 --2- 172.21.15.77:0/675177107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974107940 0x7f9974107d20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9964009930 tx=0x7f996402a420 comp rx=0 tx=0).stop 2024-05-27T08:48:19.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 -- 172.21.15.77:0/675177107 shutdown_connections 2024-05-27T08:48:19.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 --2- 172.21.15.77:0/675177107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9974101980 0x7f9974101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 --2- 172.21.15.77:0/675177107 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974102340 0x7f997410e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.126 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 --2- 172.21.15.77:0/675177107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974107940 0x7f9974107d20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 -- 172.21.15.77:0/675177107 >> 172.21.15.77:0/675177107 conn(0x7f9974078ad0 msgr2=0x7f9974078ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:19.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 -- 172.21.15.77:0/675177107 shutdown_connections 2024-05-27T08:48:19.127 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.124+0000 7f99790fc640 1 -- 172.21.15.77:0/675177107 wait complete. 2024-05-27T08:48:19.131 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f99790fc640 1 Processor -- start 2024-05-27T08:48:19.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f99790fc640 1 -- start start 2024-05-27T08:48:19.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f99790fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9974101980 0x7f9974198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f99790fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974102340 0x7f9974198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.132 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f99790fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974107940 0x7f997419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f99790fc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9974111bd0 con 0x7f9974102340 2024-05-27T08:48:19.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f99790fc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9974111a50 con 0x7f9974107940 2024-05-27T08:48:19.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f99790fc640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9974111d50 con 0x7f9974101980 2024-05-27T08:48:19.133 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f997259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974102340 0x7f9974198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f997259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974102340 0x7f9974198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48416/0 (socket says 172.21.15.77:48416) 2024-05-27T08:48:19.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.125+0000 7f997259c640 1 -- 172.21.15.77:0/4165717766 learned_addr learned my addr 172.21.15.77:0/4165717766 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:19.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f9972d9d640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9974101980 0x7f9974198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f997359e640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974107940 0x7f997419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f997259c640 1 -- 172.21.15.77:0/4165717766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9974101980 msgr2=0x7f9974198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f997259c640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9974101980 0x7f9974198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f997259c640 1 -- 172.21.15.77:0/4165717766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974107940 msgr2=0x7f997419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f997259c640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974107940 0x7f997419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f997259c640 1 -- 172.21.15.77:0/4165717766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997419d040 con 0x7f9974102340 2024-05-27T08:48:19.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f997359e640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974107940 0x7f997419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:48:19.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f9972d9d640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9974101980 0x7f9974198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:48:19.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f997259c640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974102340 0x7f9974198590 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c920 tx=0x7f995c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:19.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f996bfff640 1 -- 172.21.15.77:0/4165717766 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c007c70 con 0x7f9974102340 2024-05-27T08:48:19.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f996bfff640 1 -- 172.21.15.77:0/4165717766 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f995c007e10 con 0x7f9974102340 2024-05-27T08:48:19.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997419d330 con 0x7f9974102340 2024-05-27T08:48:19.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f996bfff640 1 -- 172.21.15.77:0/4165717766 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c018820 con 0x7f9974102340 2024-05-27T08:48:19.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.126+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99741a4bc0 con 0x7f9974102340 2024-05-27T08:48:19.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.127+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99740507e0 con 0x7f9974102340 2024-05-27T08:48:19.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.129+0000 7f996bfff640 1 -- 172.21.15.77:0/4165717766 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f995c00f450 con 0x7f9974102340 2024-05-27T08:48:19.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.129+0000 7f996bfff640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f99600b2a90 0x7f99600b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.129+0000 7f9972d9d640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f99600b2a90 0x7f99600b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.130+0000 7f996bfff640 1 -- 172.21.15.77:0/4165717766 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f995c0e2a80 con 0x7f9974102340 2024-05-27T08:48:19.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.130+0000 7f9972d9d640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f99600b2a90 0x7f99600b4f50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9964000c00 tx=0x7f9964009610 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:19.138 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.137+0000 7f996bfff640 1 -- 172.21.15.77:0/4165717766 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f995c014030 con 0x7f9974102340 2024-05-27T08:48:19.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.350+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 --> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9974062e90 con 0x7f99600b2a90 2024-05-27T08:48:19.359 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.358+0000 7f996bfff640 1 -- 172.21.15.77:0/4165717766 <== mgr.15198 v2:172.21.15.77:6800/4086463008 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9974062e90 con 0x7f99600b2a90 2024-05-27T08:48:19.360 INFO:teuthology.orchestra.run.smithi077.stdout:/ 1m 6m3h 2024-05-27T08:48:19.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f99600b2a90 msgr2=0x7f99600b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f99600b2a90 0x7f99600b4f50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9964000c00 tx=0x7f9964009610 comp rx=0 tx=0).stop 2024-05-27T08:48:19.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974102340 msgr2=0x7f9974198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974102340 0x7f9974198590 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c920 tx=0x7f995c00cde0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.362 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 shutdown_connections 2024-05-27T08:48:19.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f99600b2a90 0x7f99600b4f50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9974107940 0x7f997419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9974102340 0x7f9974198590 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 --2- 172.21.15.77:0/4165717766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9974101980 0x7f9974198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.360+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 >> 172.21.15.77:0/4165717766 conn(0x7f9974078ad0 msgr2=0x7f99741044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:19.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.361+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 shutdown_connections 2024-05-27T08:48:19.363 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.361+0000 7f99790fc640 1 -- 172.21.15.77:0/4165717766 wait complete. 2024-05-27T08:48:19.373 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-27T08:48:19.388 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-27T08:48:19.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 -- 172.21.15.77:0/3402616612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c103ce0 msgr2=0x7fb70c1040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 --2- 172.21.15.77:0/3402616612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c103ce0 0x7fb70c1040c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009ca0 tx=0x7fb6fc02a9e0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 -- 172.21.15.77:0/3402616612 shutdown_connections 2024-05-27T08:48:19.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 --2- 172.21.15.77:0/3402616612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70c10d020 0x7fb70c10f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 --2- 172.21.15.77:0/3402616612 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb70c104600 0x7fb70c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.496 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 --2- 172.21.15.77:0/3402616612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c103ce0 0x7fb70c1040c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 -- 172.21.15.77:0/3402616612 >> 172.21.15.77:0/3402616612 conn(0x7fb70c078a50 msgr2=0x7fb70c078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:19.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 -- 172.21.15.77:0/3402616612 shutdown_connections 2024-05-27T08:48:19.497 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.494+0000 7fb710e47640 1 -- 172.21.15.77:0/3402616612 wait complete. 2024-05-27T08:48:19.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb710e47640 1 Processor -- start 2024-05-27T08:48:19.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb710e47640 1 -- start start 2024-05-27T08:48:19.499 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb710e47640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70c103ce0 0x7fb70c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb710e47640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c104600 0x7fb70c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb710e47640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb70c10d020 0x7fb70c19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb710e47640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c112d20 con 0x7fb70c104600 2024-05-27T08:48:19.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb710e47640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c112ba0 con 0x7fb70c10d020 2024-05-27T08:48:19.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb710e47640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c112ea0 con 0x7fb70c103ce0 2024-05-27T08:48:19.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb709d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c104600 0x7fb70c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb709d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c104600 0x7fb70c1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48446/0 (socket says 172.21.15.77:48446) 2024-05-27T08:48:19.500 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.495+0000 7fb709d9b640 1 -- 172.21.15.77:0/846132353 learned_addr learned my addr 172.21.15.77:0/846132353 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:19.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb70a59c640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70c103ce0 0x7fb70c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb70ad9d640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb70c10d020 0x7fb70c19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb709d9b640 1 -- 172.21.15.77:0/846132353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70c103ce0 msgr2=0x7fb70c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb709d9b640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70c103ce0 0x7fb70c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb709d9b640 1 -- 172.21.15.77:0/846132353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb70c10d020 msgr2=0x7fb70c19db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb709d9b640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb70c10d020 0x7fb70c19db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb709d9b640 1 -- 172.21.15.77:0/846132353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c19e250 con 0x7fb70c104600 2024-05-27T08:48:19.501 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb70ad9d640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb70c10d020 0x7fb70c19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:48:19.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb709d9b640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c104600 0x7fb70c1997a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb70000d950 tx=0x7fb70000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:19.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb6fb7fe640 1 -- 172.21.15.77:0/846132353 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700017070 con 0x7fb70c104600 2024-05-27T08:48:19.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c19e540 con 0x7fb70c104600 2024-05-27T08:48:19.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c1a5dd0 con 0x7fb70c104600 2024-05-27T08:48:19.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb6fb7fe640 1 -- 172.21.15.77:0/846132353 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb700009c30 con 0x7fb70c104600 2024-05-27T08:48:19.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.496+0000 7fb6fb7fe640 1 -- 172.21.15.77:0/846132353 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb700020060 con 0x7fb70c104600 2024-05-27T08:48:19.502 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.497+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec005290 con 0x7fb70c104600 2024-05-27T08:48:19.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.499+0000 7fb6fb7fe640 1 -- 172.21.15.77:0/846132353 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb700005030 con 0x7fb70c104600 2024-05-27T08:48:19.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.500+0000 7fb6fb7fe640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb6f40b2960 0x7fb6f40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.500+0000 7fb6fb7fe640 1 -- 172.21.15.77:0/846132353 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fb7000e2140 con 0x7fb70c104600 2024-05-27T08:48:19.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.500+0000 7fb70a59c640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb6f40b2960 0x7fb6f40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.503 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.501+0000 7fb70a59c640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb6f40b2960 0x7fb6f40b4e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc000f80 tx=0x7fb6fc009800 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:19.504 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.502+0000 7fb6fb7fe640 1 -- 172.21.15.77:0/846132353 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7000a9b10 con 0x7fb70c104600 2024-05-27T08:48:19.696 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.694+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6ec005850 con 0x7fb70c104600 2024-05-27T08:48:19.698 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.696+0000 7fb6fb7fe640 1 -- 172.21.15.77:0/846132353 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+23144 (secure 0 0 0) 0x7fb700009de0 con 0x7fb70c104600 2024-05-27T08:48:19.698 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:48:19.699 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":94,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:48:10.631516+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/3561834991":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/4211521509":"2024-05-28T08:38:16.210207+0000","172.21.15.77:6801/2118880698":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:6800/2118880698":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/2410459695":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/2484667947":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:48:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb6f40b2960 msgr2=0x7fb6f40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb6f40b2960 0x7fb6f40b4e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc000f80 tx=0x7fb6fc009800 comp rx=0 tx=0).stop 2024-05-27T08:48:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c104600 msgr2=0x7fb70c1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.701 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c104600 0x7fb70c1997a0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb70000d950 tx=0x7fb70000de10 comp rx=0 tx=0).stop 2024-05-27T08:48:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 shutdown_connections 2024-05-27T08:48:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fb6f40b2960 0x7fb6f40b4e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb70c10d020 0x7fb70c19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.702 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb70c104600 0x7fb70c1997a0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 --2- 172.21.15.77:0/846132353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb70c103ce0 0x7fb70c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 >> 172.21.15.77:0/846132353 conn(0x7fb70c078a50 msgr2=0x7fb70c0fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 shutdown_connections 2024-05-27T08:48:19.703 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.699+0000 7fb710e47640 1 -- 172.21.15.77:0/846132353 wait complete. 2024-05-27T08:48:19.713 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-27T08:30:25.158063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-27T08:35:02.123294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-27T08:35:05.632802+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 93, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-27T08:48:19.714 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-27T08:48:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 -- 172.21.15.77:0/3627066367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 msgr2=0x7fa8d0107d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/3627066367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 0x7fa8d0107d10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009930 tx=0x7fa8c002a420 comp rx=0 tx=0).stop 2024-05-27T08:48:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 -- 172.21.15.77:0/3627066367 shutdown_connections 2024-05-27T08:48:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/3627066367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8d0101920 0x7fa8d0101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.821 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/3627066367 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8d01022e0 0x7fa8d010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/3627066367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 0x7fa8d0107d10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 -- 172.21.15.77:0/3627066367 >> 172.21.15.77:0/3627066367 conn(0x7fa8d0078a50 msgr2=0x7fa8d0078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:19.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 -- 172.21.15.77:0/3627066367 shutdown_connections 2024-05-27T08:48:19.822 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.819+0000 7fa8d4b04640 1 -- 172.21.15.77:0/3627066367 wait complete. 2024-05-27T08:48:19.824 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.820+0000 7fa8d4b04640 1 Processor -- start 2024-05-27T08:48:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.820+0000 7fa8d4b04640 1 -- start start 2024-05-27T08:48:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.820+0000 7fa8d4b04640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8d0101920 0x7fa8d0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.820+0000 7fa8d4b04640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8d01022e0 0x7fa8d01997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.820+0000 7fa8d4b04640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 0x7fa8d019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.820+0000 7fa8d4b04640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0111bf0 con 0x7fa8d0107930 2024-05-27T08:48:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.820+0000 7fa8d4b04640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0111a70 con 0x7fa8d0101920 2024-05-27T08:48:19.825 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.820+0000 7fa8d4b04640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0111d70 con 0x7fa8d01022e0 2024-05-27T08:48:19.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 0x7fa8d019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 0x7fa8d019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50110/0 (socket says 172.21.15.77:50110) 2024-05-27T08:48:19.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 -- 172.21.15.77:0/2136843291 learned_addr learned my addr 172.21.15.77:0/2136843291 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:19.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8cdd9b640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8d01022e0 0x7fa8d01997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ce59c640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8d0101920 0x7fa8d0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 -- 172.21.15.77:0/2136843291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8d01022e0 msgr2=0x7fa8d01997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8d01022e0 0x7fa8d01997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.826 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 -- 172.21.15.77:0/2136843291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8d0101920 msgr2=0x7fa8d0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:19.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8d0101920 0x7fa8d0199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:19.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 -- 172.21.15.77:0/2136843291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d019e270 con 0x7fa8d0107930 2024-05-27T08:48:19.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8ced9d640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 0x7fa8d019db40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800eb10 tx=0x7fa8c800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:19.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8bf7fe640 1 -- 172.21.15.77:0/2136843291 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8004b20 con 0x7fa8d0107930 2024-05-27T08:48:19.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8bf7fe640 1 -- 172.21.15.77:0/2136843291 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8c8004cc0 con 0x7fa8d0107930 2024-05-27T08:48:19.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d019e560 con 0x7fa8d0107930 2024-05-27T08:48:19.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8bf7fe640 1 -- 172.21.15.77:0/2136843291 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8010670 con 0x7fa8d0107930 2024-05-27T08:48:19.827 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.821+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d01a5df0 con 0x7fa8d0107930 2024-05-27T08:48:19.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.822+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d0069870 con 0x7fa8d0107930 2024-05-27T08:48:19.828 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.824+0000 7fa8bf7fe640 1 -- 172.21.15.77:0/2136843291 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa8c8005000 con 0x7fa8d0107930 2024-05-27T08:48:19.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.825+0000 7fa8bf7fe640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa8b80b2960 0x7fa8b80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:19.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.825+0000 7fa8ce59c640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa8b80b2960 0x7fa8b80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:19.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.825+0000 7fa8bf7fe640 1 -- 172.21.15.77:0/2136843291 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fa8c80e2d30 con 0x7fa8d0107930 2024-05-27T08:48:19.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.826+0000 7fa8ce59c640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa8b80b2960 0x7fa8b80b4e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009900 tx=0x7fa8c0000bc0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:19.830 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:19.829+0000 7fa8bf7fe640 1 -- 172.21.15.77:0/2136843291 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8c8054df0 con 0x7fa8d0107930 2024-05-27T08:48:19.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:19 smithi092 ceph-mon[36049]: pgmap v312: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 684 KiB/s wr, 45 op/s 2024-05-27T08:48:20.018 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.017+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa8d002dba0 con 0x7fa8d0107930 2024-05-27T08:48:20.019 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.017+0000 7fa8bf7fe640 1 -- 172.21.15.77:0/2136843291 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fa8c80aa700 con 0x7fa8d0107930 2024-05-27T08:48:20.019 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 1 2024-05-27T08:48:20.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa8b80b2960 msgr2=0x7fa8b80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa8b80b2960 0x7fa8b80b4e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009900 tx=0x7fa8c0000bc0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 msgr2=0x7fa8d019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 0x7fa8d019db40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800eb10 tx=0x7fa8c800c2d0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 shutdown_connections 2024-05-27T08:48:20.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa8b80b2960 0x7fa8b80b4e20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa8d01022e0 0x7fa8d01997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8d0107930 0x7fa8d019db40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 --2- 172.21.15.77:0/2136843291 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa8d0101920 0x7fa8d0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 >> 172.21.15.77:0/2136843291 conn(0x7fa8d0078a50 msgr2=0x7fa8d006a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:20.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.019+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 shutdown_connections 2024-05-27T08:48:20.022 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.020+0000 7fa8d4b04640 1 -- 172.21.15.77:0/2136843291 wait complete. 2024-05-27T08:48:20.031 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:19 smithi077 ceph-mon[32007]: pgmap v312: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 684 KiB/s wr, 45 op/s 2024-05-27T08:48:20.032 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-27T08:48:20.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:19 smithi179 ceph-mon[36481]: pgmap v312: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 684 KiB/s wr, 45 op/s 2024-05-27T08:48:20.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 -- 172.21.15.77:0/317764385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 msgr2=0x7f942410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 --2- 172.21.15.77:0/317764385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 0x7f942410e7f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f941c009cd0 tx=0x7f941c02a8e0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 -- 172.21.15.77:0/317764385 shutdown_connections 2024-05-27T08:48:20.140 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 --2- 172.21.15.77:0/317764385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9424101920 0x7f9424101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 --2- 172.21.15.77:0/317764385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 0x7f942410e7f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 --2- 172.21.15.77:0/317764385 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9424107930 0x7f9424107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 -- 172.21.15.77:0/317764385 >> 172.21.15.77:0/317764385 conn(0x7f9424078a50 msgr2=0x7f9424078e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:20.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 -- 172.21.15.77:0/317764385 shutdown_connections 2024-05-27T08:48:20.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 -- 172.21.15.77:0/317764385 wait complete. 2024-05-27T08:48:20.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 Processor -- start 2024-05-27T08:48:20.141 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.136+0000 7f942c159640 1 -- start start 2024-05-27T08:48:20.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942c159640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9424101920 0x7f9424199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942c159640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 0x7f942419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942c159640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9424107930 0x7f94241a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942c159640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9424111bf0 con 0x7f94241022e0 2024-05-27T08:48:20.142 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942c159640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9424111a70 con 0x7f9424107930 2024-05-27T08:48:20.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942c159640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9424111d70 con 0x7f9424101920 2024-05-27T08:48:20.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 0x7f942419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 0x7f942419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50126/0 (socket says 172.21.15.77:50126) 2024-05-27T08:48:20.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 -- 172.21.15.77:0/1571968714 learned_addr learned my addr 172.21.15.77:0/1571968714 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:20.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f9429ef5640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9424101920 0x7f9424199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942a6f6640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9424107930 0x7f94241a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.143 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 -- 172.21.15.77:0/1571968714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9424101920 msgr2=0x7f9424199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9424101920 0x7f9424199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 -- 172.21.15.77:0/1571968714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9424107930 msgr2=0x7f94241a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9424107930 0x7f94241a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 -- 172.21.15.77:0/1571968714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94241a5590 con 0x7f94241022e0 2024-05-27T08:48:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942a6f6640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9424107930 0x7f94241a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-27T08:48:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f94296f4640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 0x7f942419a510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f941400e9f0 tx=0x7f941400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f941affd640 1 -- 172.21.15.77:0/1571968714 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941400ccc0 con 0x7f94241022e0 2024-05-27T08:48:20.144 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f941affd640 1 -- 172.21.15.77:0/1571968714 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f941400ce60 con 0x7f94241022e0 2024-05-27T08:48:20.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94241a5760 con 0x7f94241022e0 2024-05-27T08:48:20.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.137+0000 7f941affd640 1 -- 172.21.15.77:0/1571968714 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9414010430 con 0x7f94241022e0 2024-05-27T08:48:20.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.138+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94241a5cb0 con 0x7f94241022e0 2024-05-27T08:48:20.145 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.139+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9424069870 con 0x7f94241022e0 2024-05-27T08:48:20.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.141+0000 7f941affd640 1 -- 172.21.15.77:0/1571968714 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f94140046c0 con 0x7f94241022e0 2024-05-27T08:48:20.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.141+0000 7f941affd640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94100b29e0 0x7f94100b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.142+0000 7f9429ef5640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94100b29e0 0x7f94100b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.142+0000 7f941affd640 1 -- 172.21.15.77:0/1571968714 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f94140e2c30 con 0x7f94241022e0 2024-05-27T08:48:20.146 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.142+0000 7f9429ef5640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94100b29e0 0x7f94100b4ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f942000ab60 tx=0x7f942000a2d0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:20.147 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.145+0000 7f941affd640 1 -- 172.21.15.77:0/1571968714 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9414054df0 con 0x7f94241022e0 2024-05-27T08:48:20.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.333+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9424109ee0 con 0x7f94241022e0 2024-05-27T08:48:20.335 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.334+0000 7f941affd640 1 -- 172.21.15.77:0/1571968714 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f94140cc020 con 0x7f94241022e0 2024-05-27T08:48:20.335 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 32 2024-05-27T08:48:20.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94100b29e0 msgr2=0x7f94100b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94100b29e0 0x7f94100b4ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f942000ab60 tx=0x7f942000a2d0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.337 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 msgr2=0x7f942419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 0x7f942419a510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f941400e9f0 tx=0x7f941400edf0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 shutdown_connections 2024-05-27T08:48:20.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f94100b29e0 0x7f94100b4ea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9424107930 0x7f94241a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f94241022e0 0x7f942419a510 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 --2- 172.21.15.77:0/1571968714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9424101920 0x7f9424199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.338 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 >> 172.21.15.77:0/1571968714 conn(0x7f9424078a50 msgr2=0x7f94241040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:20.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 shutdown_connections 2024-05-27T08:48:20.339 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.336+0000 7f942c159640 1 -- 172.21.15.77:0/1571968714 wait complete. 2024-05-27T08:48:20.348 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-27T08:48:20.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 -- 172.21.15.77:0/1816471745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b601022d0 msgr2=0x7f5b601026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.455 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 --2- 172.21.15.77:0/1816471745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b601022d0 0x7f5b601026b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009900 tx=0x7f5b5002a410 comp rx=0 tx=0).stop 2024-05-27T08:48:20.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 -- 172.21.15.77:0/1816471745 shutdown_connections 2024-05-27T08:48:20.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 --2- 172.21.15.77:0/1816471745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b60106e80 0x7f5b6010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 --2- 172.21.15.77:0/1816471745 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b60102c80 0x7f5b601067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 --2- 172.21.15.77:0/1816471745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b601022d0 0x7f5b601026b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 -- 172.21.15.77:0/1816471745 >> 172.21.15.77:0/1816471745 conn(0x7f5b60077f10 msgr2=0x7f5b600ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:20.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 -- 172.21.15.77:0/1816471745 shutdown_connections 2024-05-27T08:48:20.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.452+0000 7f5b6681d640 1 -- 172.21.15.77:0/1816471745 wait complete. 2024-05-27T08:48:20.456 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b6681d640 1 Processor -- start 2024-05-27T08:48:20.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b6681d640 1 -- start start 2024-05-27T08:48:20.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b6681d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b601022d0 0x7f5b6010f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b6681d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b60106e80 0x7f5b6010f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b6681d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6010fe60 0x7f5b601a7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b6681d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b60112f40 con 0x7f5b6010fe60 2024-05-27T08:48:20.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b6681d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5b60112dc0 con 0x7f5b601022d0 2024-05-27T08:48:20.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b6681d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b601130c0 con 0x7f5b60106e80 2024-05-27T08:48:20.457 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6010fe60 0x7f5b601a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6010fe60 0x7f5b601a7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50134/0 (socket says 172.21.15.77:50134) 2024-05-27T08:48:20.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 -- 172.21.15.77:0/2629183314 learned_addr learned my addr 172.21.15.77:0/2629183314 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:20.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b57fff640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b60106e80 0x7f5b6010f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b5ffff640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b601022d0 0x7f5b6010f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 -- 172.21.15.77:0/2629183314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b60106e80 msgr2=0x7f5b6010f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b60106e80 0x7f5b6010f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 -- 172.21.15.77:0/2629183314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b601022d0 msgr2=0x7f5b6010f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.458 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b601022d0 0x7f5b6010f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 -- 172.21.15.77:0/2629183314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b601a7d90 con 0x7f5b6010fe60 2024-05-27T08:48:20.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b57fff640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b60106e80 0x7f5b6010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:48:20.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.453+0000 7f5b64dba640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6010fe60 0x7f5b601a7690 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800b970 tx=0x7f5b5800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:20.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.454+0000 7f5b5dffb640 1 -- 172.21.15.77:0/2629183314 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58019070 con 0x7f5b6010fe60 2024-05-27T08:48:20.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.454+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b601a8080 con 0x7f5b6010fe60 2024-05-27T08:48:20.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.454+0000 7f5b5dffb640 1 -- 172.21.15.77:0/2629183314 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b58014440 con 0x7f5b6010fe60 2024-05-27T08:48:20.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.454+0000 7f5b5dffb640 1 -- 172.21.15.77:0/2629183314 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b580075d0 con 0x7f5b6010fe60 2024-05-27T08:48:20.459 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.454+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b601a8580 con 0x7f5b6010fe60 2024-05-27T08:48:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.455+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b60069cc0 con 0x7f5b6010fe60 2024-05-27T08:48:20.460 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.456+0000 7f5b5dffb640 1 -- 172.21.15.77:0/2629183314 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5b5801c070 con 0x7f5b6010fe60 2024-05-27T08:48:20.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.457+0000 7f5b5dffb640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5b4c0b2990 0x7f5b4c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.457+0000 7f5b5ffff640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5b4c0b2990 0x7f5b4c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.457+0000 7f5b5dffb640 1 -- 172.21.15.77:0/2629183314 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f5b580e1fd0 con 0x7f5b6010fe60 2024-05-27T08:48:20.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.457+0000 7f5b5ffff640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5b4c0b2990 0x7f5b4c0b4e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b50031440 tx=0x7f5b500095f0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:20.461 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.460+0000 7f5b5dffb640 1 -- 172.21.15.77:0/2629183314 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b580a9ab0 con 0x7f5b6010fe60 2024-05-27T08:48:20.644 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.643+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5b6006ae70 con 0x7f5b6010fe60 2024-05-27T08:48:20.645 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.643+0000 7f5b5dffb640 1 -- 172.21.15.77:0/2629183314 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f5b580a93b0 con 0x7f5b6010fe60 2024-05-27T08:48:20.645 INFO:teuthology.orchestra.run.smithi077.stdout:pg_num: 64 2024-05-27T08:48:20.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5b4c0b2990 msgr2=0x7f5b4c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5b4c0b2990 0x7f5b4c0b4e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5b50031440 tx=0x7f5b500095f0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6010fe60 msgr2=0x7f5b601a7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6010fe60 0x7f5b601a7690 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800b970 tx=0x7f5b5800be30 comp rx=0 tx=0).stop 2024-05-27T08:48:20.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 shutdown_connections 2024-05-27T08:48:20.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5b4c0b2990 0x7f5b4c0b4e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b60106e80 0x7f5b6010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6010fe60 0x7f5b601a7690 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 --2- 172.21.15.77:0/2629183314 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5b601022d0 0x7f5b6010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 >> 172.21.15.77:0/2629183314 conn(0x7f5b60077f10 msgr2=0x7f5b601051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:20.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 shutdown_connections 2024-05-27T08:48:20.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.646+0000 7f5b6681d640 1 -- 172.21.15.77:0/2629183314 wait complete. 2024-05-27T08:48:20.659 DEBUG:teuthology.orchestra.run.smithi077:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-27T08:48:20.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 -- 172.21.15.77:0/1279625081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 msgr2=0x7f209c108e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 --2- 172.21.15.77:0/1279625081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 0x7f209c108e20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f208c009930 tx=0x7f208c02a420 comp rx=0 tx=0).stop 2024-05-27T08:48:20.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 -- 172.21.15.77:0/1279625081 shutdown_connections 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 --2- 172.21.15.77:0/1279625081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f209c1033f0 0x7f209c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 --2- 172.21.15.77:0/1279625081 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c102a30 0x7f209c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 --2- 172.21.15.77:0/1279625081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 0x7f209c108e20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 -- 172.21.15.77:0/1279625081 >> 172.21.15.77:0/1279625081 conn(0x7f209c079e50 msgr2=0x7f209c07a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 -- 172.21.15.77:0/1279625081 shutdown_connections 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 -- 172.21.15.77:0/1279625081 wait complete. 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 Processor -- start 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.759+0000 7f20a24fc640 1 -- start start 2024-05-27T08:48:20.763 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a24fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c102a30 0x7f209c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a24fc640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f209c1033f0 0x7f209c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a24fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 0x7f209c19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a24fc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f209c112d00 con 0x7f209c108a40 2024-05-27T08:48:20.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a24fc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f209c112b80 con 0x7f209c102a30 2024-05-27T08:48:20.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a24fc640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f209c112e80 con 0x7f209c1033f0 2024-05-27T08:48:20.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 0x7f209c19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 0x7f209c19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:50164/0 (socket says 172.21.15.77:50164) 2024-05-27T08:48:20.764 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 -- 172.21.15.77:0/3854693004 learned_addr learned my addr 172.21.15.77:0/3854693004 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:20.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f209bfff640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c102a30 0x7f209c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f209b7fe640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f209c1033f0 0x7f209c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 -- 172.21.15.77:0/3854693004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f209c1033f0 msgr2=0x7f209c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f209c1033f0 0x7f209c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 -- 172.21.15.77:0/3854693004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c102a30 msgr2=0x7f209c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:20.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c102a30 0x7f209c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:20.765 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 -- 172.21.15.77:0/3854693004 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c19e270 con 0x7f209c108a40 2024-05-27T08:48:20.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20a0a99640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 0x7f209c19db40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f209400cad0 tx=0x7f209400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:20.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.760+0000 7f20997fa640 1 -- 172.21.15.77:0/3854693004 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209400ae20 con 0x7f209c108a40 2024-05-27T08:48:20.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.761+0000 7f20997fa640 1 -- 172.21.15.77:0/3854693004 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2094005020 con 0x7f209c108a40 2024-05-27T08:48:20.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.761+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c19e560 con 0x7f209c108a40 2024-05-27T08:48:20.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.761+0000 7f20997fa640 1 -- 172.21.15.77:0/3854693004 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209400f6b0 con 0x7f209c108a40 2024-05-27T08:48:20.766 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.761+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c1a5df0 con 0x7f209c108a40 2024-05-27T08:48:20.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.762+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c069cc0 con 0x7f209c108a40 2024-05-27T08:48:20.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.764+0000 7f20997fa640 1 -- 172.21.15.77:0/3854693004 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f20940051d0 con 0x7f209c108a40 2024-05-27T08:48:20.767 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.764+0000 7f20997fa640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f20840b2960 0x7f20840b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:20.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.764+0000 7f209bfff640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f20840b2960 0x7f20840b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:20.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.765+0000 7f20997fa640 1 -- 172.21.15.77:0/3854693004 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f20940e2a40 con 0x7f209c108a40 2024-05-27T08:48:20.768 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.765+0000 7f209bfff640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f20840b2960 0x7f20840b4e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f208c000c00 tx=0x7f208c032ca0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:20.769 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.768+0000 7f20997fa640 1 -- 172.21.15.77:0/3854693004 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2094014030 con 0x7f209c108a40 2024-05-27T08:48:20.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:20 smithi092 ceph-mon[36049]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:48:20.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:20 smithi092 ceph-mon[36049]: from='client.15807 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:48:20.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:20 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/846132353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:48:20.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:20 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2136843291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:48:20.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:20 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1571968714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:48:20.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:20 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2629183314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:48:20.998 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:20.997+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f209c10aa10 con 0x7f209c108a40 2024-05-27T08:48:21.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.006+0000 7f20997fa640 1 -- 172.21.15.77:0/3854693004 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816175 (secure 0 0 0) 0x7f20940aa410 con 0x7f209c108a40 2024-05-27T08:48:21.017 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:48:21.017 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:48:21.017 INFO:teuthology.orchestra.run.smithi077.stdout: "epoch": 26, 2024-05-27T08:48:21.018 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.018 INFO:teuthology.orchestra.run.smithi077.stdout: "active_gid": 15198, 2024-05-27T08:48:21.018 INFO:teuthology.orchestra.run.smithi077.stdout: "active_name": "x", 2024-05-27T08:48:21.018 INFO:teuthology.orchestra.run.smithi077.stdout: "active_addrs": { 2024-05-27T08:48:21.018 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-05-27T08:48:21.018 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.018 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-05-27T08:48:21.018 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:6800", 2024-05-27T08:48:21.019 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 4086463008 2024-05-27T08:48:21.019 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.019 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.019 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v1", 2024-05-27T08:48:21.019 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:6801", 2024-05-27T08:48:21.019 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 4086463008 2024-05-27T08:48:21.019 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.019 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:21.020 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.020 INFO:teuthology.orchestra.run.smithi077.stdout: "active_addr": "172.21.15.77:6801/4086463008", 2024-05-27T08:48:21.020 INFO:teuthology.orchestra.run.smithi077.stdout: "active_change": "2024-05-27T08:38:16.210421+0000", 2024-05-27T08:48:21.020 INFO:teuthology.orchestra.run.smithi077.stdout: "active_mgr_features": 4540701547738038271, 2024-05-27T08:48:21.020 INFO:teuthology.orchestra.run.smithi077.stdout: "available": true, 2024-05-27T08:48:21.020 INFO:teuthology.orchestra.run.smithi077.stdout: "standbys": [ 2024-05-27T08:48:21.020 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.020 INFO:teuthology.orchestra.run.smithi077.stdout: "gid": 24802, 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "z", 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_features": 4540701547738038271, 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.021 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:21.022 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:21.022 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.022 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.022 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.022 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:21.022 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.022 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.022 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.023 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:48:21.023 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.023 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.023 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.023 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.023 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.023 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.023 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.024 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.025 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.025 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.025 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.025 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.025 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.025 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.025 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.025 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.026 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.026 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.026 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.026 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.026 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.026 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.026 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.026 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.027 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.027 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.027 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.027 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.027 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.027 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.027 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.028 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.029 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.029 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.029 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.029 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.029 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.029 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.029 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.029 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.030 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.030 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.030 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.030 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.030 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.030 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.030 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.031 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.031 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.031 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.031 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.031 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.031 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.031 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.031 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.032 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.033 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.033 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.033 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:48:21.033 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.033 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.033 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.033 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.033 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-05-27T08:48:21.034 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-05-27T08:48:21.034 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.034 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.034 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.034 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-05-27T08:48:21.034 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.034 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.034 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.035 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-05-27T08:48:21.035 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.035 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.035 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.035 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-05-27T08:48:21.035 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-05-27T08:48:21.035 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.036 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.036 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.036 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.036 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.036 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.036 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.036 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-05-27T08:48:21.036 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.037 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.037 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.037 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.037 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-05-27T08:48:21.037 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-05-27T08:48:21.037 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.037 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.037 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.038 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.038 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.038 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.038 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.038 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-05-27T08:48:21.038 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.038 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.038 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.039 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.039 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-05-27T08:48:21.039 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-05-27T08:48:21.039 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.039 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.039 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.039 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-05-27T08:48:21.039 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.040 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.040 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.040 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-05-27T08:48:21.040 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.040 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.040 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.040 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.040 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-05-27T08:48:21.041 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-05-27T08:48:21.041 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.041 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.041 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.041 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.041 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.041 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.041 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.042 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:48:21.042 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.042 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.042 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.042 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.042 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-05-27T08:48:21.042 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-05-27T08:48:21.043 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.043 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.043 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.043 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.043 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.043 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.043 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.043 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:48:21.044 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.044 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.044 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.044 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.044 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-05-27T08:48:21.044 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-05-27T08:48:21.044 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.044 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-05-27T08:48:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.045 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.046 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.047 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.047 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.047 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.047 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.047 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.047 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.047 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.047 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.048 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.048 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.048 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.048 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-05-27T08:48:21.048 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.048 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.048 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.048 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.049 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:48:21.050 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.050 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.050 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.050 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.050 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-05-27T08:48:21.050 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-05-27T08:48:21.050 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.050 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.051 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.051 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-05-27T08:48:21.051 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.051 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.051 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.051 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:48:21.051 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:48:21.051 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.052 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.052 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.052 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-05-27T08:48:21.052 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-05-27T08:48:21.052 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.052 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.052 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.052 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.053 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-05-27T08:48:21.053 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-05-27T08:48:21.053 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.053 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:48:21.053 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:48:21.053 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.053 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.053 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.054 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-05-27T08:48:21.054 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:48:21.054 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.054 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.054 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.054 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-05-27T08:48:21.054 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:21.054 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-05-27T08:48:21.055 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.055 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:48:21.055 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.055 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.055 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.055 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.055 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-05-27T08:48:21.055 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-05-27T08:48:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:48:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.056 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:48:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:48:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-05-27T08:48:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-05-27T08:48:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.057 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.058 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.058 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-05-27T08:48:21.058 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-05-27T08:48:21.058 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-05-27T08:48:21.058 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.058 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:48:21.058 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:48:21.058 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-05-27T08:48:21.059 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.060 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.060 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.060 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:48:21.060 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:48:21.060 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.060 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.060 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.060 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.061 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:48:21.062 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:48:21.062 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.062 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.062 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.062 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.062 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.062 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.062 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.063 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.063 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.063 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.063 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.063 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.063 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.063 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.063 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.064 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.065 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.065 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.065 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.065 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.065 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.065 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.065 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.065 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.066 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.066 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.066 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.066 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.066 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.066 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.066 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.066 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.067 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.067 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.067 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.067 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.067 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.067 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.067 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.067 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.068 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.069 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.069 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.069 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.069 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.069 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.069 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.069 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.069 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.070 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.070 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.070 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.070 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.070 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.070 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.070 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.070 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.071 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.071 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-05-27T08:48:21.071 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-05-27T08:48:21.071 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.071 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.071 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.071 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.071 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-05-27T08:48:21.072 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-05-27T08:48:21.073 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.073 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.073 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.073 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-05-27T08:48:21.073 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.073 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.073 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.073 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-05-27T08:48:21.074 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-05-27T08:48:21.074 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-05-27T08:48:21.074 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-05-27T08:48:21.074 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-05-27T08:48:21.074 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.074 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-05-27T08:48:21.074 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.074 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.075 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.075 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.075 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-05-27T08:48:21.075 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-05-27T08:48:21.075 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.075 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.075 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.075 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.076 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.076 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.076 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.076 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:48:21.076 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.076 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.076 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.076 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.077 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:21.077 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:21.077 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.077 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.077 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.077 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:21.077 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.077 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:48:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.078 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.079 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.080 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.080 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.080 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.080 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-05-27T08:48:21.080 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-05-27T08:48:21.080 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.080 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.080 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.081 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:21.081 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:21.081 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.081 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.081 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:48:21.081 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:48:21.081 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.081 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.082 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.082 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-05-27T08:48:21.082 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:48:21.082 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.082 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.082 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.082 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.082 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.083 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.084 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.084 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-05-27T08:48:21.084 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.084 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.084 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.084 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-05-27T08:48:21.084 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-05-27T08:48:21.084 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.085 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.086 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.086 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.086 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-05-27T08:48:21.086 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-05-27T08:48:21.086 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.086 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.086 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.087 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-05-27T08:48:21.087 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.087 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.087 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.087 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:48:21.087 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.087 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.087 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-05-27T08:48:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-05-27T08:48:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-05-27T08:48:21.088 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.089 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.089 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.089 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:48:21.089 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.089 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.089 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.089 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.089 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-05-27T08:48:21.090 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-05-27T08:48:21.090 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.090 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.090 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.090 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.090 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.090 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.090 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.091 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.092 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.092 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:21.092 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.092 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.092 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.092 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:48:21.093 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.093 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.093 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.093 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.093 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:48:21.093 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:48:21.093 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.093 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.094 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.095 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.095 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-05-27T08:48:21.095 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-05-27T08:48:21.095 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.095 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.095 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.095 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.095 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.096 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.096 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.096 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-05-27T08:48:21.096 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-05-27T08:48:21.096 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-05-27T08:48:21.096 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.096 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:48:21.096 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.097 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.098 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.098 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.098 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.098 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:48:21.098 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.098 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.098 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.098 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.099 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.100 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:48:21.100 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.100 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.100 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.100 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.100 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-05-27T08:48:21.100 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-05-27T08:48:21.100 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.101 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.101 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.101 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.101 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.101 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.101 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.101 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:48:21.101 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-05-27T08:48:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-05-27T08:48:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.102 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.103 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.104 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-05-27T08:48:21.104 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-05-27T08:48:21.104 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.104 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.104 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.104 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:48:21.104 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.104 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:48:21.105 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.106 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.106 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.106 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:48:21.106 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.106 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.106 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.106 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-05-27T08:48:21.107 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.107 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.107 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.107 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.107 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-05-27T08:48:21.107 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-05-27T08:48:21.107 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.107 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.108 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.108 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:48:21.108 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.108 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.108 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.108 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:21.108 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.108 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.109 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.109 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.109 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-05-27T08:48:21.109 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-05-27T08:48:21.109 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.109 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.109 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.109 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:48:21.110 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.110 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.110 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.110 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-05-27T08:48:21.110 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.110 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.110 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.110 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.111 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-05-27T08:48:21.111 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:48:21.111 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.111 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.111 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.111 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:48:21.111 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.111 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:48:21.112 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.113 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.113 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.113 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:48:21.113 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.113 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.113 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.113 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:48:21.113 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.114 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.114 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.114 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.114 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-05-27T08:48:21.114 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:48:21.114 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.114 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.114 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.115 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.116 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-05-27T08:48:21.116 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-05-27T08:48:21.116 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.116 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.116 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.116 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:48:21.116 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.116 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.117 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.117 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-05-27T08:48:21.117 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.117 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.117 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.117 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.117 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-05-27T08:48:21.117 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-05-27T08:48:21.118 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.119 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.119 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.119 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.119 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-05-27T08:48:21.119 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-05-27T08:48:21.119 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.119 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.119 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.120 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.121 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-05-27T08:48:21.121 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-05-27T08:48:21.121 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.121 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.121 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.121 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:48:21.121 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.121 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.122 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.122 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-05-27T08:48:21.122 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.122 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.122 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.122 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-05-27T08:48:21.122 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-05-27T08:48:21.123 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.123 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.123 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.123 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:48:21.123 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.123 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.123 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.124 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.125 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:48:21.125 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.125 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.125 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.125 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-05-27T08:48:21.125 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.125 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.125 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.126 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.126 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-05-27T08:48:21.126 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-05-27T08:48:21.126 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.126 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.126 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.126 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:48:21.126 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.127 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.127 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.127 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:48:21.127 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.127 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.127 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.127 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.127 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-05-27T08:48:21.128 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:48:21.128 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.128 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.128 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.128 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:48:21.128 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.128 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.128 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.129 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.130 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.130 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.130 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.130 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.130 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.130 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:48:21.130 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.130 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.131 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.131 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.131 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-05-27T08:48:21.131 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:48:21.131 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.131 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.131 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.131 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.132 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:48:21.133 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:48:21.133 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.133 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.133 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.133 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-05-27T08:48:21.133 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.133 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.133 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.134 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:48:21.134 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.134 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.134 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.134 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-05-27T08:48:21.134 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:48:21.135 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.136 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.136 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.136 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-05-27T08:48:21.136 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-05-27T08:48:21.136 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.136 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.136 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.138 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-05-27T08:48:21.138 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-05-27T08:48:21.138 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.138 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.138 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.138 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.138 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.138 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.139 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.139 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:48:21.139 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.139 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.139 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.139 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-05-27T08:48:21.139 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:48:21.140 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.141 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.141 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.141 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-05-27T08:48:21.141 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:48:21.141 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.141 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.141 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.142 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:48:21.142 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.142 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.142 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.142 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:48:21.142 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.142 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.142 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.143 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.144 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.144 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:48:21.144 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.144 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.144 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.144 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-05-27T08:48:21.144 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:48:21.145 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.146 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.146 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.146 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-05-27T08:48:21.146 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-05-27T08:48:21.146 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.146 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.146 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.147 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.147 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.147 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.147 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.147 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:48:21.147 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.147 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.148 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.149 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.149 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.149 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.149 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.149 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.149 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.149 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.149 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-05-27T08:48:21.150 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.151 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.151 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.151 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.151 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.151 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.151 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.151 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:48:21.151 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.152 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.153 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.153 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.153 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.153 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:48:21.153 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.153 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.153 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.153 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.154 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.154 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.154 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.154 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.154 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.154 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.154 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.154 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.155 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.156 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.156 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.156 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.156 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.156 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.156 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.156 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.157 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.158 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.158 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:48:21.158 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:48:21.158 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.158 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.158 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.158 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.158 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.159 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.159 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.159 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:48:21.159 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.159 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.159 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.159 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:48:21.160 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:48:21.160 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.160 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.160 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.160 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-05-27T08:48:21.160 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.160 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.160 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.161 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.162 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.162 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.162 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.162 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.162 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.162 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:48:21.162 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.162 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.163 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.163 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-05-27T08:48:21.163 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-05-27T08:48:21.163 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.163 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.163 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.164 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.165 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-05-27T08:48:21.165 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-05-27T08:48:21.165 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.165 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.165 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.165 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.165 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.165 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.166 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.166 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-05-27T08:48:21.166 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.166 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.166 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.166 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-05-27T08:48:21.166 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-05-27T08:48:21.167 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-05-27T08:48:21.168 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.168 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:48:21.168 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.168 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.168 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.168 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-05-27T08:48:21.168 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-05-27T08:48:21.169 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.169 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.169 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.169 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-05-27T08:48:21.169 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.169 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.169 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.170 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:48:21.170 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.170 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.170 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.170 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.170 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-05-27T08:48:21.170 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:48:21.170 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.171 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.172 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.172 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.172 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-05-27T08:48:21.172 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-05-27T08:48:21.172 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.172 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.172 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.172 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.173 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.173 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.173 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.173 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:48:21.173 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.173 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.173 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.174 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.175 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:48:21.175 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.175 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.175 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.175 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-05-27T08:48:21.175 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-05-27T08:48:21.175 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.176 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.176 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.176 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.176 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.176 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.176 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.176 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:48:21.176 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.177 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.178 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.178 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.178 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.178 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:48:21.178 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.178 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.178 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.179 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-05-27T08:48:21.179 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:48:21.179 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.179 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.179 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.179 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.179 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.179 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.180 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.180 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:48:21.180 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.180 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.180 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.180 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.180 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-05-27T08:48:21.180 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-05-27T08:48:21.181 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.181 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.181 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.181 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-05-27T08:48:21.181 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.181 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.181 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.181 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:48:21.182 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.182 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.182 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.182 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.182 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.182 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.182 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.182 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.183 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.183 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.183 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.183 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.183 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.183 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.183 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.183 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.184 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.184 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.184 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-05-27T08:48:21.184 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-05-27T08:48:21.184 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.184 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.184 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.184 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.185 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-05-27T08:48:21.186 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-05-27T08:48:21.186 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.186 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.186 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.186 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.186 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.186 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.186 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.187 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:48:21.187 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.187 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.187 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.187 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.187 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-05-27T08:48:21.187 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-05-27T08:48:21.187 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:48:21.188 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.189 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.190 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.190 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.190 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.190 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:48:21.190 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.190 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.190 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.191 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.191 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-05-27T08:48:21.191 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:48:21.191 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.191 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.191 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.191 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.191 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.192 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.192 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.192 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:48:21.192 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.192 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.192 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.192 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.193 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-05-27T08:48:21.193 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:48:21.193 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.193 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.193 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.193 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.193 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.193 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.194 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.194 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:48:21.194 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.194 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.194 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.194 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.194 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.194 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-05-27T08:48:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.195 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.196 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.197 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.197 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.197 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.197 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.197 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.197 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.197 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.197 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.198 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.198 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.198 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.198 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.198 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.198 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.198 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.198 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.199 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.200 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.201 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.202 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.202 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.202 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.202 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.202 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.202 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.202 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.203 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.203 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.203 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.203 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.203 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.203 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.203 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.203 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.204 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.205 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.206 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.206 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.206 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.206 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.206 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.206 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.206 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.206 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.207 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.207 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.207 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.207 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-05-27T08:48:21.207 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-05-27T08:48:21.207 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.207 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.207 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.208 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-05-27T08:48:21.208 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.208 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.208 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.208 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:48:21.208 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.208 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.208 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.209 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.209 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.209 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.209 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.209 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-05-27T08:48:21.209 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.209 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.209 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.210 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:48:21.210 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:48:21.210 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.210 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.210 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.210 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.210 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.210 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.211 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.211 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.211 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.211 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.211 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.211 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.211 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:48:21.211 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:48:21.212 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.212 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.212 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.212 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.212 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.212 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.212 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.212 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.213 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.213 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.213 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.213 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.213 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:48:21.213 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:48:21.213 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.213 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.214 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.214 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.214 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.214 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.214 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.214 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.214 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.214 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.215 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.215 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.215 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:48:21.215 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:48:21.215 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.215 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.215 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.215 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.216 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.216 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.216 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.216 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.216 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.216 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.216 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.216 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.217 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:48:21.217 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:48:21.217 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.217 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.217 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.217 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.217 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.217 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:48:21.218 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.219 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.219 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.219 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.219 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.219 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.219 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.219 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.219 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.220 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.220 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.220 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.220 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:48:21.220 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:48:21.220 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.220 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.220 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.221 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.222 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:48:21.222 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:48:21.222 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.222 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.222 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.222 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.222 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.222 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:48:21.223 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.224 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.225 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.225 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.225 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.225 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:48:21.225 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:48:21.225 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.225 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.225 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.226 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.227 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:48:21.227 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:48:21.227 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.227 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.227 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.227 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.227 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.227 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.228 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.228 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.228 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.228 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.228 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.228 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.228 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:48:21.228 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.229 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.230 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.230 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.230 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.230 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:48:21.230 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:48:21.230 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.230 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.230 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.231 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.232 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:48:21.232 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:48:21.232 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.232 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.232 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.232 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.232 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.232 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:48:21.233 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.234 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.234 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.234 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:21.234 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.234 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.234 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.234 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.234 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.235 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.235 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.235 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.235 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:48:21.235 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:48:21.235 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.235 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.236 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.237 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.237 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-05-27T08:48:21.237 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:48:21.237 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.237 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.237 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.237 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.237 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:48:21.238 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:48:21.239 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.239 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.239 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.239 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:21.239 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.239 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.239 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.239 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.240 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.241 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.242 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.242 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:48:21.242 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:48:21.242 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.242 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.242 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.242 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.242 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.243 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.243 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.243 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.243 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.243 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.243 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.243 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.243 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.244 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:48:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:48:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.245 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.246 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.246 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.246 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.246 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.246 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.246 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.246 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.246 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.247 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.247 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.247 INFO:teuthology.orchestra.run.smithi077.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:48:21.247 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:48:21.247 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.247 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.247 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.247 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "[]", 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.248 INFO:teuthology.orchestra.run.smithi077.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:48:21.249 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:48:21.249 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.249 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.249 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.249 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "{}", 2024-05-27T08:48:21.249 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.249 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.249 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.250 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.251 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.251 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.251 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.251 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.251 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.251 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.251 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.251 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.252 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.253 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.253 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.253 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.253 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.253 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.253 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.253 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.253 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:48:21.254 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:48:21.254 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.254 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.254 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.254 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.254 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.254 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.254 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.255 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.255 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.255 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.255 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.255 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.255 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:48:21.255 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:48:21.255 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.256 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.257 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.257 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.257 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:48:21.257 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:48:21.257 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.257 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.257 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.258 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.258 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.258 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.258 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.258 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.258 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.258 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.258 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.259 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.260 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.260 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.260 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.260 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.260 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.260 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.260 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:48:21.260 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.261 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.262 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.262 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.262 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.262 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:48:21.262 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:48:21.262 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.262 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.263 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.264 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.264 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:48:21.264 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:48:21.264 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.264 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.264 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.264 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.264 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.265 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.265 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.265 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.265 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.265 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.265 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.265 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.265 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:48:21.266 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:48:21.266 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.266 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.266 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.266 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.266 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.266 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.266 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.267 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.268 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.268 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:48:21.268 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.268 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.268 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.268 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.268 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.268 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.269 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.270 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.270 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.270 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.270 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.270 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.270 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.270 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.271 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:48:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:48:21.272 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.273 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:48:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:48:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.274 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.275 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.276 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.277 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.277 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.277 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.277 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.277 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.277 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:48:21.277 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:48:21.277 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.278 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.278 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.278 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.278 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.278 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.278 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.278 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.278 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.279 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.279 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.279 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.279 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:48:21.279 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:48:21.279 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.279 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.280 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.280 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.280 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.280 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.280 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.280 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.280 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.280 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.281 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.282 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:48:21.283 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:48:21.283 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.283 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.283 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.283 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.283 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.283 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.283 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.284 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.284 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.284 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.284 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.284 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.284 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:48:21.284 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:48:21.284 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.285 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.285 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.285 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.285 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.285 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.285 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.285 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.285 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.286 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.286 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.286 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:48:21.286 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:48:21.286 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.286 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.286 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.287 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:21.287 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.287 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.287 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.287 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.287 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.287 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.287 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.288 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.289 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.289 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.289 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.289 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.289 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.289 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.289 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-05-27T08:48:21.289 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-05-27T08:48:21.290 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.290 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.290 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.290 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.290 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.290 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.290 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.290 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.291 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.292 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.292 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.292 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.292 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.292 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:48:21.292 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.293 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-05-27T08:48:21.294 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.294 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.294 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.294 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.294 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.294 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.294 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.294 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-05-27T08:48:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-05-27T08:48:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.295 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.296 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.297 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.297 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.297 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.297 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.297 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.297 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.297 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.297 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.298 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.298 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.298 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.298 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.298 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.298 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.298 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.298 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.299 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.300 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.300 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.300 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.300 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.300 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.300 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.300 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.301 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.302 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.302 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.302 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.302 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.302 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.302 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.302 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.302 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.303 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.303 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.303 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.303 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.303 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.303 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.303 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.304 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.305 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.305 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-05-27T08:48:21.305 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-05-27T08:48:21.305 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.305 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.305 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.305 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.305 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.306 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.306 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.306 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-05-27T08:48:21.306 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.306 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.306 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.306 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.307 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.308 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.308 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.308 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.308 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.308 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:21.308 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:21.308 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.308 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.309 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.309 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-05-27T08:48:21.309 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.309 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.309 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.309 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.309 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.309 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.310 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.310 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:21.310 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:21.310 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.310 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.310 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.310 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.311 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.312 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.312 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.312 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.312 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.312 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.312 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.312 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.312 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.313 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.314 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.314 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.314 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.314 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.314 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.314 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.314 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.314 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.315 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.315 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-05-27T08:48:21.315 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-05-27T08:48:21.315 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.315 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.315 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.315 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.316 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-05-27T08:48:21.317 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-05-27T08:48:21.317 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.317 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.317 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.317 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-05-27T08:48:21.317 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.317 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.317 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.318 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-05-27T08:48:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-05-27T08:48:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-05-27T08:48:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-05-27T08:48:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-05-27T08:48:21.319 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.320 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.320 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.320 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.320 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.320 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.320 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-05-27T08:48:21.320 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-05-27T08:48:21.320 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.321 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.322 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.322 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.322 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.322 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.322 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-05-27T08:48:21.322 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.322 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.323 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.323 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-05-27T08:48:21.323 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-05-27T08:48:21.323 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.323 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.323 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.323 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.323 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.324 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.325 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.325 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.325 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.325 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.325 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.325 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.325 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.325 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.326 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.326 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.326 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.326 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.326 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.326 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.326 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.327 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.328 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.329 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.330 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.330 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.330 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.330 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.330 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.330 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.330 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.330 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.331 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.331 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.331 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.331 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.331 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.331 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.331 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.331 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.332 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.333 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.333 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.333 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-05-27T08:48:21.333 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-05-27T08:48:21.333 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.333 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.333 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-05-27T08:48:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:48:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.334 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.335 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.336 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.336 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:48:21.336 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.336 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.336 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.336 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.336 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-05-27T08:48:21.336 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-05-27T08:48:21.337 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.337 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.337 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.337 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-05-27T08:48:21.337 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.337 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.337 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.337 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:48:21.338 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.338 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.338 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.338 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.338 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-05-27T08:48:21.338 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-05-27T08:48:21.338 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.338 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.339 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.339 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-05-27T08:48:21.339 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.339 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.339 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.342 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:48:21.342 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.342 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.342 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.342 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.343 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-05-27T08:48:21.343 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-05-27T08:48:21.343 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.343 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.343 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.343 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.343 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.343 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.344 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.344 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:48:21.344 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.344 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.344 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.344 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.344 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:21.344 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:21.345 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.345 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.345 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.345 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:21.345 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.345 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.345 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.345 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:48:21.346 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.346 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.346 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.346 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.346 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.346 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.346 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.346 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.347 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.347 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.347 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.347 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.347 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.347 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.347 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.347 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-05-27T08:48:21.348 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.349 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.349 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.349 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:48:21.349 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.349 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.349 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.349 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.349 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.350 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.350 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-05-27T08:48:21.350 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.350 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.350 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.350 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.351 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.351 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.351 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.351 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.351 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.351 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.351 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.352 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.352 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.352 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.352 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.352 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.352 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.352 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.352 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.353 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.353 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.353 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.353 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.353 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.353 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.353 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.353 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.354 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.354 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.354 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.354 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.354 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.354 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.354 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.354 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.355 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.355 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.355 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.355 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.355 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.355 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.355 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.355 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.356 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.356 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.356 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.356 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.356 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.356 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.356 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.356 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.357 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.357 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.357 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.357 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.357 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.357 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.357 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.357 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.358 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.358 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.358 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.358 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.358 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.358 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.358 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.358 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.359 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.360 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.360 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-05-27T08:48:21.360 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.360 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.360 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.360 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.360 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.360 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-05-27T08:48:21.361 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.362 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.362 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.362 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.362 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.362 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.362 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.362 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.362 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:21.363 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:21.363 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.363 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.363 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.363 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:21.363 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.363 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.363 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.364 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.365 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-05-27T08:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-05-27T08:48:21.366 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.367 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.368 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.368 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:48:21.368 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.368 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.368 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.368 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.368 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-05-27T08:48:21.368 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:48:21.369 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.370 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.370 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.370 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.370 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-05-27T08:48:21.370 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-05-27T08:48:21.370 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.370 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.370 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.371 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.371 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.371 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.371 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.371 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:48:21.371 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.371 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.371 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-05-27T08:48:21.372 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.373 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.373 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:48:21.373 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.373 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.373 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.373 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.373 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.373 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.374 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.374 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.374 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.374 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.374 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.374 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.374 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.375 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.375 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.375 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.375 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.375 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.375 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.375 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.375 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.376 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.376 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.376 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.376 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.376 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.376 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.376 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.377 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.378 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.378 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.378 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.378 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.378 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.378 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.378 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.378 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.379 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.379 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.379 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.379 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.379 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.379 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.379 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.379 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.380 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.381 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.381 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.381 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.381 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.381 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.381 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.381 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.381 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.382 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.382 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.382 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-05-27T08:48:21.382 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-05-27T08:48:21.382 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.382 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.382 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.383 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.384 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-05-27T08:48:21.384 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-05-27T08:48:21.384 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.384 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.384 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.384 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-05-27T08:48:21.384 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.384 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.385 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.386 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.386 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.386 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.386 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.386 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.386 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.386 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.386 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.387 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.387 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.387 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.387 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-05-27T08:48:21.387 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-05-27T08:48:21.387 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.387 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.387 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.388 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.389 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-05-27T08:48:21.389 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-05-27T08:48:21.389 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.389 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.389 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.389 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:21.389 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:21.389 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-05-27T08:48:21.390 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.390 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:48:21.390 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.390 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.390 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.390 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.390 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-05-27T08:48:21.390 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:48:21.391 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.392 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.392 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.392 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.392 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-05-27T08:48:21.392 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-05-27T08:48:21.392 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.392 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.392 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.393 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-05-27T08:48:21.393 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.393 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.393 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.393 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:48:21.393 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.393 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.393 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.394 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.395 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.395 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.395 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.395 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.395 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.395 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.395 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.395 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.396 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.396 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.396 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.396 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.396 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.396 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.396 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.396 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.397 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.397 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.397 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.397 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.397 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.397 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.397 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.398 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.398 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.398 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.398 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.398 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.398 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.398 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.398 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.399 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.399 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.399 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.399 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.399 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.399 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.399 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.399 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.400 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.400 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.400 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.400 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.400 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.400 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.400 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.400 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.401 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.402 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.402 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.402 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.402 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.402 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.402 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.402 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.402 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.403 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.403 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.403 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.403 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.403 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.403 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.403 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.404 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.404 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.404 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.404 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.404 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.404 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.404 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.404 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.405 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.406 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.406 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.406 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.406 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.406 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.406 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.406 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.406 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.407 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.407 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.407 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.407 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.407 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.407 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.407 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.407 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.408 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.409 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.409 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.409 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.409 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.409 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.409 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.409 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.409 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.410 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.411 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.412 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.412 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.412 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.412 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.412 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.412 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.412 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.412 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.413 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.413 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.413 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.413 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.413 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.413 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.413 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.413 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.414 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.415 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.415 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.415 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.415 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.415 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.415 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.415 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.415 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-05-27T08:48:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.416 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.417 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.418 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:48:21.418 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.418 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.418 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.418 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.418 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-05-27T08:48:21.418 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-05-27T08:48:21.418 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.419 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.419 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.419 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.419 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-05-27T08:48:21.419 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.419 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.419 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:48:21.419 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.420 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.420 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.420 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.420 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.420 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.420 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.420 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.420 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.421 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.421 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.421 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.421 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.421 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.421 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.421 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.421 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.422 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.422 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.422 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.422 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.422 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.422 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.422 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.422 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.423 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.423 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.423 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.423 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.423 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.423 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.423 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.423 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.424 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.425 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.425 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.425 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.425 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.425 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.425 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.425 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.425 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.426 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.426 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.426 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.426 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.426 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.426 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.426 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.426 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.427 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.427 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.427 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.427 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.427 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.427 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.427 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.428 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.428 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.428 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.428 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.428 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.428 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.428 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.428 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.429 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.429 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.429 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.429 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.429 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.429 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.429 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.429 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.430 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.430 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.430 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.430 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.430 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.430 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.430 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.430 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.431 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.431 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-05-27T08:48:21.431 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.431 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.431 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.431 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-05-27T08:48:21.431 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-05-27T08:48:21.431 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.432 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.432 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.432 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-05-27T08:48:21.432 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.432 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.432 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.432 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:48:21.432 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.433 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.434 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.434 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.434 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.434 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.434 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.434 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.434 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.434 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.435 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.435 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.435 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.435 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.435 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.435 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.435 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.435 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.436 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.437 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.437 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.437 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.437 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.437 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.437 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.437 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.437 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.438 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.438 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.438 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.438 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.438 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.438 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.438 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.438 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.439 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.440 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.440 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.440 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.440 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.440 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.440 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.440 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.440 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.441 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.441 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.441 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.441 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.441 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.441 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.441 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-05-27T08:48:21.441 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:48:21.442 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:48:21.443 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.443 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.443 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-05-27T08:48:21.443 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-05-27T08:48:21.443 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.443 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.444 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.444 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-05-27T08:48:21.444 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.444 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.444 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.444 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:48:21.444 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.444 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.445 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.445 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-05-27T08:48:21.445 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-05-27T08:48:21.445 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.445 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.445 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.445 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.446 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-05-27T08:48:21.447 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-05-27T08:48:21.447 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.447 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.447 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.447 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.447 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.447 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.447 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.448 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:48:21.448 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.448 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.448 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.448 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.448 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.448 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.448 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.449 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.450 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.450 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.450 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-05-27T08:48:21.450 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-05-27T08:48:21.450 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.450 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.450 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.450 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-05-27T08:48:21.451 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.451 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.451 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.451 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:48:21.451 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:48:21.451 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.451 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.451 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.452 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.453 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.453 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.453 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.453 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.453 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.453 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.453 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.453 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.454 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.454 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.454 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.454 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.454 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.454 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.454 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.454 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.455 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.456 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.456 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.456 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.456 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.456 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.456 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.456 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.456 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.457 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.457 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.457 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.457 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.457 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.457 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.457 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.457 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.458 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.459 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.459 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.459 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.459 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.459 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.459 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.459 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.459 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.460 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.460 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.460 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.460 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.460 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.460 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.460 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.460 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.461 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.462 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.462 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.462 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.462 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.462 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.462 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.462 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.462 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.463 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.463 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.463 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-05-27T08:48:21.463 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.463 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.463 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.463 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.463 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.464 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.464 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.464 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.464 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.464 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.464 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.464 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.464 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.465 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.466 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.466 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.466 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.466 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.466 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.466 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.466 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.467 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.467 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.467 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.467 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.467 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.467 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.467 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.467 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.468 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.468 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.468 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.468 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.468 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.468 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.468 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.468 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.469 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.470 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.470 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.470 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.470 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.470 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.470 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.470 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.470 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.471 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.472 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.472 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.472 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.472 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.472 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.472 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.472 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.472 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.473 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.474 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.474 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.474 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-05-27T08:48:21.474 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.474 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.474 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.474 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.474 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.475 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.475 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.475 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.475 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.475 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.475 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.475 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.475 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.476 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.477 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.477 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.477 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.477 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.477 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.477 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.477 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.477 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.478 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.478 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.478 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.478 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.478 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.478 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.478 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.478 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.479 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.480 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.480 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.480 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.480 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.480 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.480 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.480 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.480 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.481 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.482 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.482 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.482 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.482 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.482 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.482 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.482 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.482 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.483 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.484 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.485 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.486 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.486 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.486 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.486 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.486 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.486 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:48:21.486 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.486 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.487 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.487 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.487 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.487 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.487 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.487 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.487 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.487 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.488 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.489 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.489 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.489 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.489 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.489 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.489 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.489 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.489 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.490 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.490 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.490 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.490 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.490 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.490 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.490 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.491 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.492 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.492 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.492 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.492 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.492 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.492 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.492 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.492 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.493 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.494 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.494 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.494 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.494 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.494 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.494 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.494 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.494 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.495 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.495 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.495 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.495 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.495 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.495 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.495 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.495 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-05-27T08:48:21.496 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-05-27T08:48:21.496 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.496 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.496 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.496 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.496 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.496 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.496 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.497 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-05-27T08:48:21.497 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-05-27T08:48:21.497 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-05-27T08:48:21.497 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.497 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-05-27T08:48:21.497 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.497 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.497 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.498 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.498 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.498 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.498 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.498 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.498 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.498 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.498 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.499 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.499 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.499 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.499 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.499 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.499 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.499 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.499 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.500 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.500 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.500 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-05-27T08:48:21.500 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.500 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.500 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.500 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.500 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.501 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.502 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.502 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.502 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.502 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.502 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.502 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.502 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.502 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.503 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.503 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.503 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.503 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.503 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.503 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.503 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.503 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.504 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.504 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.504 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.504 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.504 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.504 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.504 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.504 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.505 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.505 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.505 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.505 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.505 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.505 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.505 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.505 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.506 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.506 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.506 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.506 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.506 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.506 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.506 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.506 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.507 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.508 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.509 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.509 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.509 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.509 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.509 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.509 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.509 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.509 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.510 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.511 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.511 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.511 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-05-27T08:48:21.511 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.511 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.511 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.511 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.511 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.512 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.513 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.513 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.513 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.513 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.513 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.513 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.513 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.513 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.514 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.514 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.514 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.514 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.514 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.514 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.514 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.515 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.516 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.516 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.516 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.516 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.516 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.516 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.516 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.516 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.517 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.517 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.517 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.517 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.517 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.517 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.517 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.517 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.518 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.518 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.518 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.518 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.518 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.518 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.518 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.518 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.519 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.519 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.519 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.519 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.519 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.519 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.519 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.519 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.520 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.520 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.520 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.520 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.520 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.520 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.520 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.520 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.521 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.522 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.522 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.522 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.522 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-05-27T08:48:21.522 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.522 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.522 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.523 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.524 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.524 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.524 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.524 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.524 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.524 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.524 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.524 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.525 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.526 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.526 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.526 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.526 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.526 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.526 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.526 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.526 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.527 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.527 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.527 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.527 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.527 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.527 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.527 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.527 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.528 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.528 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.528 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.528 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.528 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.528 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.528 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.528 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.529 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.530 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.530 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.530 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.530 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.530 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.530 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.530 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.530 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.531 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.531 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.531 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:21.531 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:21.531 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.531 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.531 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.532 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.533 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.533 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.533 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.533 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.533 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.533 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.533 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.533 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-05-27T08:48:21.534 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.535 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.535 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.535 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-05-27T08:48:21.535 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-05-27T08:48:21.535 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.535 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.535 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-05-27T08:48:21.535 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.536 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.537 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.537 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-05-27T08:48:21.537 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:48:21.537 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.537 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.537 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.537 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.538 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-05-27T08:48:21.539 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-05-27T08:48:21.539 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.539 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.539 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.539 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.539 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.539 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.539 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.540 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.540 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.540 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.540 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.540 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.540 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.540 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.541 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.542 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.542 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.542 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.542 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.542 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.542 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.542 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.542 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.543 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.543 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.543 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.543 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.543 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.546 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.546 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.546 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.547 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.547 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.547 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.547 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.547 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.547 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.547 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.548 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.549 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.549 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.549 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.549 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.549 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.549 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.549 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.549 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.550 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.551 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.551 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.551 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.551 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.551 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.551 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.551 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.552 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-05-27T08:48:21.552 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-05-27T08:48:21.552 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.552 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.552 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.552 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-05-27T08:48:21.552 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.552 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.553 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.553 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:48:21.553 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.553 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.553 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.553 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.553 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:21.553 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:21.554 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.554 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.554 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.554 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:21.554 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.554 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.554 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.554 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:48:21.555 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.555 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.555 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.555 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.555 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.555 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.555 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.555 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.556 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.557 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.557 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.557 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.557 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-05-27T08:48:21.557 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.557 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.557 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.558 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-05-27T08:48:21.558 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-05-27T08:48:21.558 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.558 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.558 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.558 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.558 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.558 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.559 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.559 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.559 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.559 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.559 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.559 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-05-27T08:48:21.559 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-05-27T08:48:21.560 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.560 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.560 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.560 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.560 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.560 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.560 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.560 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.561 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.562 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.562 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.562 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.562 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.562 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.562 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.562 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.562 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.563 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.563 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.563 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.563 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.563 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.563 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.564 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.564 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.564 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.564 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.564 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.564 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.564 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.565 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.565 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.565 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.565 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.565 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.565 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.565 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.566 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.566 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.566 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.566 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.566 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.566 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.566 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.566 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.567 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.568 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.568 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.568 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.568 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.568 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.568 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.568 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.569 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-05-27T08:48:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-05-27T08:48:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.570 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.571 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.571 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.571 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.571 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.571 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.571 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.571 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.572 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:48:21.572 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:48:21.572 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.572 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.572 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.572 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-05-27T08:48:21.572 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.572 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.573 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.573 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.573 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.573 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.573 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.573 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.573 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-05-27T08:48:21.573 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.574 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.575 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.575 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.575 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.575 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:21.575 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:21.575 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.575 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.575 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.576 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-05-27T08:48:21.576 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.576 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.576 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.576 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:48:21.576 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.576 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.576 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.577 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.577 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:21.577 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:21.577 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.577 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.577 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.577 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-05-27T08:48:21.577 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.578 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.578 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.578 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:48:21.578 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.578 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.578 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.578 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.578 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.579 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.579 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.579 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.579 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.579 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.579 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.579 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.579 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.580 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.581 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.581 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-05-27T08:48:21.581 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.581 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.581 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.581 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.581 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.581 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.582 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.582 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.582 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-05-27T08:48:21.582 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-05-27T08:48:21.582 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.582 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.582 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.582 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-05-27T08:48:21.583 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.583 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.583 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.583 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-05-27T08:48:21.583 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-05-27T08:48:21.583 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.583 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.583 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.584 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-05-27T08:48:21.585 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-05-27T08:48:21.585 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-05-27T08:48:21.585 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.585 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.585 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.585 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.585 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.585 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.586 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.586 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.586 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.586 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-05-27T08:48:21.586 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.586 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.586 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.587 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.588 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.588 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.588 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.588 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.588 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.588 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.588 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.588 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.589 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.589 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.589 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.589 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.589 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.589 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.589 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.589 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.590 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.591 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.591 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.591 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.591 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.591 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.591 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.591 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.591 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.592 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.592 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.592 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.592 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.592 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.592 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.592 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.592 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.593 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.593 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.593 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.593 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.593 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.593 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.593 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.593 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.594 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.594 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.594 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.594 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.594 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.594 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.594 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.595 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.595 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.595 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.595 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.595 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.595 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-05-27T08:48:21.595 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:48:21.595 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.596 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.597 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.597 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.597 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:48:21.597 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:48:21.597 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.597 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.597 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.597 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.598 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.599 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.599 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.599 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.599 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.599 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.599 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.599 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.599 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.600 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.600 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.600 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.600 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.600 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.600 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.600 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-05-27T08:48:21.600 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-05-27T08:48:21.601 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.601 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.601 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.601 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.601 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.601 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.601 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.601 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.602 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.602 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.602 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.602 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.602 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.602 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.602 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.602 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-05-27T08:48:21.603 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.603 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.603 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.603 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-05-27T08:48:21.603 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-05-27T08:48:21.603 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.603 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.603 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.604 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.604 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.604 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.604 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.604 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.604 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.604 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.604 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.605 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.606 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.606 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.606 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.606 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.606 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.606 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.606 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.606 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.607 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.607 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.607 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.607 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.607 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.607 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.607 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.607 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.608 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.608 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.608 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.608 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.608 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.608 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.608 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.608 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.609 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.610 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.610 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.610 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.610 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.610 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.610 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.610 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.611 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.611 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.611 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.611 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.611 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.611 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.611 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.611 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.612 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.612 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.612 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.612 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.612 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.612 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.612 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.612 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.613 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.613 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.613 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.613 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.613 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.613 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.613 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.613 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.614 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.614 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.614 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.614 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.614 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.614 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.614 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.614 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.615 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.615 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.615 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.615 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:21.615 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:21.615 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.615 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.616 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.616 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.616 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.616 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.616 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.616 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.616 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.616 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.617 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.617 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.617 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:21.617 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:21.617 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.617 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.617 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.617 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.618 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.618 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.618 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.618 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.618 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.618 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.618 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.619 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.620 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.620 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.620 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.620 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.620 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.620 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.620 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.620 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-05-27T08:48:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.621 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.622 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.622 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.622 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.622 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.622 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.622 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.622 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.622 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.623 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.623 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.623 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.623 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.623 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.623 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.623 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.623 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.624 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.624 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.624 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.624 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.624 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.624 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.624 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.624 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.625 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.625 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.625 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.625 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.625 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.625 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.625 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.626 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.626 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.626 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.626 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.626 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.626 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.626 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.627 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.627 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.627 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.627 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.627 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.627 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.627 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.627 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.628 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.629 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.629 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.629 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.629 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.629 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.629 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.629 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.629 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.630 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.631 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.631 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.631 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.631 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.631 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.631 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.631 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.631 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.632 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.632 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.632 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.632 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-05-27T08:48:21.632 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.632 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.632 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.632 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.633 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.634 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.634 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.634 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.634 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.634 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.634 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.634 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.635 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.635 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.635 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.635 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.635 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.635 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.635 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.635 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.636 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.636 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.636 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.636 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.636 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.636 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.636 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.636 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.637 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.637 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.637 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.637 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.637 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.637 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.637 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.638 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.638 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.638 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.638 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.638 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.638 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.638 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.638 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.639 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.639 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.639 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.639 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.639 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.639 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.639 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.640 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.640 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.640 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.640 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.640 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.640 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.640 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.640 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.641 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.642 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.642 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.642 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.642 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.642 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.642 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.642 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.642 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.643 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.643 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.643 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-05-27T08:48:21.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-05-27T08:48:21.643 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.643 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.643 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.644 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.645 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.645 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.645 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.645 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-05-27T08:48:21.645 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.645 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.645 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.645 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.646 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.646 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.646 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.646 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.646 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.646 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.646 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.646 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.647 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.647 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.647 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.647 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.647 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.647 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.647 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.647 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.648 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.648 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.648 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.648 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.648 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.648 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.648 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.648 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.649 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.649 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.649 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.649 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.649 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.649 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.649 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.649 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.650 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.650 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.650 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.650 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.650 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.650 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.650 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.651 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.651 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.651 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.651 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.651 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.651 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.651 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.651 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.652 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.652 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.652 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.652 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.652 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.652 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.652 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.652 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.653 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.654 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.654 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.654 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.654 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-05-27T08:48:21.654 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-05-27T08:48:21.654 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.655 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.656 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-05-27T08:48:21.657 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.657 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.657 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.657 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.657 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.657 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.657 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.658 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-05-27T08:48:21.658 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-05-27T08:48:21.658 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.658 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.658 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.658 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.658 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.659 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.659 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.659 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.659 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.659 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.659 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.659 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.659 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.660 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.661 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.661 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.661 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.661 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-05-27T08:48:21.661 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-05-27T08:48:21.661 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.661 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.662 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.662 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.662 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.662 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.662 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.662 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.662 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.662 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.663 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.663 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.663 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-05-27T08:48:21.663 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-05-27T08:48:21.663 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.663 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.663 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.663 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.664 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-05-27T08:48:21.665 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-05-27T08:48:21.665 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.665 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.665 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.665 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.665 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.665 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.665 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.666 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.666 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.666 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.666 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.666 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.666 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-05-27T08:48:21.666 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-05-27T08:48:21.666 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.667 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.667 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.667 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.667 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.667 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.667 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.667 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.667 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.668 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.669 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:21.669 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-05-27T08:48:21.669 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.669 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.669 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.669 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.669 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.669 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.670 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.670 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.670 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.670 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.670 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.670 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.670 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.670 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.671 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.671 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.671 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.671 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.671 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.671 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.671 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-05-27T08:48:21.671 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.672 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.673 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.673 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.673 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.673 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-05-27T08:48:21.673 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-05-27T08:48:21.673 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.673 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.673 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.674 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.674 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.674 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.674 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.674 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.674 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.674 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.674 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.675 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.675 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-05-27T08:48:21.675 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-05-27T08:48:21.675 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.675 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.675 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.675 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.675 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.676 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.676 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.676 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.676 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.676 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.676 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.676 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.676 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.677 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.677 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.677 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smb", 2024-05-27T08:48:21.677 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.677 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.677 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.677 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.677 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.678 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.678 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.678 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.678 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.678 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.678 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.678 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.678 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.679 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.680 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.680 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.680 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.680 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.680 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.680 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.680 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.680 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.681 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.681 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.681 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.681 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.681 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.681 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.681 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.681 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.682 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.682 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.682 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.682 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.682 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.682 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.683 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.683 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.683 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.683 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.683 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.683 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.683 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.683 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.684 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.685 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.685 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.685 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.685 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.685 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.685 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.685 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.685 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.686 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.686 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.686 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.686 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.686 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.686 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.686 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.687 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.687 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.687 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.687 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.687 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.687 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.687 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.687 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: "update_orchestration": { 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "update_orchestration", 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.688 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.689 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.689 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.689 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.689 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:48:21.689 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.689 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.689 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.689 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.690 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.690 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.690 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.690 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-05-27T08:48:21.690 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.690 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.690 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.691 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-05-27T08:48:21.691 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-05-27T08:48:21.691 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.691 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.691 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.691 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.691 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.691 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.692 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.692 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:48:21.692 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.692 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.692 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.692 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.692 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-05-27T08:48:21.692 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:48:21.693 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.694 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.694 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.694 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.694 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.694 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.694 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.694 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.695 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.695 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.695 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.695 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.695 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.695 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.695 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.695 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.696 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.696 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.696 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.696 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.696 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.696 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.696 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.696 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.697 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.697 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.697 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.697 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.697 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.697 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.697 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.697 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.698 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.699 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.700 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.701 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.702 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.702 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.702 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.702 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.702 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.702 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.702 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.702 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.703 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.703 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.703 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.703 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.703 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.703 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.703 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.703 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.704 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.704 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.704 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.704 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.704 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.704 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.704 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.704 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.705 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.706 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.706 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.706 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.706 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.706 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.706 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.706 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.707 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.707 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.707 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.707 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.707 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.707 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.707 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.707 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.708 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.709 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.709 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.709 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.709 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.709 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.709 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.709 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.710 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.710 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.710 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.710 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.710 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.710 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.710 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.710 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.711 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.711 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.711 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.711 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.711 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.711 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.711 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.711 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.712 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.712 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.712 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.712 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.712 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.712 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.712 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.712 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.713 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.713 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.713 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.713 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.713 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.713 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.713 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.713 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.714 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.714 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.714 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.714 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.714 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.714 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.714 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.714 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.715 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.715 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.715 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.715 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.715 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.715 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.715 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.715 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.716 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.717 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.718 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.718 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.718 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.718 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.718 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.718 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.718 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.719 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.720 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.720 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.720 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.720 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.720 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.720 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.720 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.720 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.721 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.721 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.721 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.721 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.721 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.721 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.721 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.721 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.722 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.723 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.723 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.723 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.723 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.723 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.723 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.723 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.723 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.724 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.724 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.724 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.724 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.724 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.724 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.724 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.724 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.725 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.725 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.725 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.725 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.725 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.725 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.725 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.725 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.726 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.726 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.726 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.726 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.726 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.726 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.726 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.726 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.727 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-05-27T08:48:21.728 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-05-27T08:48:21.728 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.728 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.728 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.728 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:48:21.728 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.728 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.728 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.729 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.730 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.730 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-05-27T08:48:21.730 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.730 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.730 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.730 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.730 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.731 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.731 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.731 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.731 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.731 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.731 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.731 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.731 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.732 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.732 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.732 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.732 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.732 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.732 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.732 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.732 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.733 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.733 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.733 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.733 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.733 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.733 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.733 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.733 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.734 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.735 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.735 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.735 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.735 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.735 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.735 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.735 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.736 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.736 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.736 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.736 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.736 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.736 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.736 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.736 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.737 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.738 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.738 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.738 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.738 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.738 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.738 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.738 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.739 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.739 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.739 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.739 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.739 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.739 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.739 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.739 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.740 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.740 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.740 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.740 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.740 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.740 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.740 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.740 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.741 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-05-27T08:48:21.742 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.742 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.742 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.742 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-05-27T08:48:21.742 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-05-27T08:48:21.742 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.742 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.742 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.743 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.743 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.743 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.743 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.743 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:48:21.743 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.743 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.743 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.744 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.745 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.745 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:48:21.745 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.745 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.745 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.745 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.745 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-05-27T08:48:21.745 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-05-27T08:48:21.746 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.746 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.746 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.746 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.746 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.746 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.746 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.746 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.747 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.748 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.748 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.748 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.748 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.748 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:48:21.748 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.748 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.748 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.749 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:48:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-05-27T08:48:21.750 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-05-27T08:48:21.751 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.751 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.751 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.754 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.754 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.754 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.754 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.754 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.754 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.754 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.755 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.755 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.755 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-05-27T08:48:21.755 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-05-27T08:48:21.755 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.755 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.755 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.755 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.756 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.756 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.756 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.756 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.756 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.756 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.756 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.756 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.757 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-05-27T08:48:21.757 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-05-27T08:48:21.757 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.757 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.757 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.757 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:48:21.757 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.757 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.758 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.758 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.758 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.758 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.758 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.758 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.758 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-05-27T08:48:21.758 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-05-27T08:48:21.759 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.759 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.759 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.759 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.759 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.759 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.759 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.759 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.760 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.760 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.760 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.760 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.760 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:21.760 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:21.760 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.760 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.761 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.761 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-05-27T08:48:21.761 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-05-27T08:48:21.761 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.761 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.761 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.761 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.761 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.762 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.762 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.762 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-05-27T08:48:21.762 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-05-27T08:48:21.762 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.762 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.762 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.762 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-05-27T08:48:21.763 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.763 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.763 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.763 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.763 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.763 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.763 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.763 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.764 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.765 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.765 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.765 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.765 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.765 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.765 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-05-27T08:48:21.765 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-05-27T08:48:21.765 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.766 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.766 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.766 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.766 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.766 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.766 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.766 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.766 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.767 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.767 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.767 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.767 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.767 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.767 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.767 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.767 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.768 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.768 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.768 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.768 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.768 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.768 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.768 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.768 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.769 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.769 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.769 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.769 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.769 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.769 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.769 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.769 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.770 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.770 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.770 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.770 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.770 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.770 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.770 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.770 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.771 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.772 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.772 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.772 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.772 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.772 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.772 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.772 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.772 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.774 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.774 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.774 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.774 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.774 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.774 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.774 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.774 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.775 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.775 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.775 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.775 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.775 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.775 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.775 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.775 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-05-27T08:48:21.776 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-05-27T08:48:21.776 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.776 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.776 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.776 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.776 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.776 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.776 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.777 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.778 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.778 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.778 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.778 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.778 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.778 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.778 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.778 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.779 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.779 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.779 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.779 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.779 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.779 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.779 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.780 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.780 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.780 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.780 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.780 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.780 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.780 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.780 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.781 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.781 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-05-27T08:48:21.781 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-05-27T08:48:21.781 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.781 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.781 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.781 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:48:21.781 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.782 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.782 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.782 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.782 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.782 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.782 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.782 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.782 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.783 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.783 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.783 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-05-27T08:48:21.783 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.783 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.783 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.783 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.783 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.784 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.784 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.784 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.784 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.784 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.784 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.784 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.784 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.785 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.785 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.785 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.785 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.785 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.785 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.785 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.785 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.786 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.787 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.787 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.787 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.787 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.787 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.787 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.787 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.787 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.788 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.788 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.788 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.788 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.788 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.788 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.788 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.788 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.789 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.790 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.790 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.790 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.790 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.790 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.790 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.790 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.790 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.791 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.791 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.791 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.791 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.791 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.791 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.791 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.791 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.792 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.792 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.792 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.792 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.792 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.792 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.792 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.792 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.793 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.793 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.793 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.793 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.793 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.793 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.793 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.793 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.794 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.794 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.794 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.794 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-05-27T08:48:21.794 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.794 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.794 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.794 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.795 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.795 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.795 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.795 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.795 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.795 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.795 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.795 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.796 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.797 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.797 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.797 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.797 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.797 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.797 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.797 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.797 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.798 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.798 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.798 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.798 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.798 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.798 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.798 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.798 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.799 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.799 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.799 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.799 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.799 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.799 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.799 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.799 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.800 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.801 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.801 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.801 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.801 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.801 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.801 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.801 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.801 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.802 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.803 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.803 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-05-27T08:48:21.803 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-05-27T08:48:21.803 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.803 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.803 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.804 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-05-27T08:48:21.804 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.804 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.804 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.804 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:48:21.804 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.804 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.804 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.805 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-05-27T08:48:21.805 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-05-27T08:48:21.805 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.805 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.805 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.805 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.805 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.806 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.806 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.806 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-05-27T08:48:21.806 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.806 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.806 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.806 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.806 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.807 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:48:21.808 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.808 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.808 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.808 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.808 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:48:21.808 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:48:21.808 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.808 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.809 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.809 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.809 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.809 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.809 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.809 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:48:21.809 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.809 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.810 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.810 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.810 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.810 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.810 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.810 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.810 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.810 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.811 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.811 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.811 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.811 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.811 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.811 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.811 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.811 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.812 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.812 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.812 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-05-27T08:48:21.812 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.812 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.812 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.812 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.813 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.814 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.814 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.814 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.814 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-05-27T08:48:21.814 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-05-27T08:48:21.814 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.814 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.815 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.815 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.815 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.815 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.815 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.815 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.815 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.815 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.816 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.816 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.816 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:21.816 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:21.816 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.816 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.816 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.816 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:21.817 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.817 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.817 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.817 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.817 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.817 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.817 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.817 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.818 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.819 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.819 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.819 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.819 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.819 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.819 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.819 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.819 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.820 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.820 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.820 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.820 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.820 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.820 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.820 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.820 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.821 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.821 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.821 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.821 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.821 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.821 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.821 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.821 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.822 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.823 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.823 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.823 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.823 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.823 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.823 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.823 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.823 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.824 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.824 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.824 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.824 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.824 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.824 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.824 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.824 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.825 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.826 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.826 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.826 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.826 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.826 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.826 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.826 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.827 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.827 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.827 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.827 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.827 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.827 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.827 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.827 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.828 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.828 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.828 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-05-27T08:48:21.828 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-05-27T08:48:21.828 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.828 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.828 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.828 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.829 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-05-27T08:48:21.830 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-05-27T08:48:21.830 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.830 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.830 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.830 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-05-27T08:48:21.830 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.830 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.830 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.831 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.831 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.831 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.831 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.831 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.831 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-05-27T08:48:21.831 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-05-27T08:48:21.831 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.832 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.832 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.832 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:48:21.832 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.832 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.832 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.832 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.832 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.833 INFO:teuthology.orchestra.run.smithi077.stdout: "gid": 24806, 2024-05-27T08:48:21.834 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "y", 2024-05-27T08:48:21.834 INFO:teuthology.orchestra.run.smithi077.stdout: "mgr_features": 4540701547738038271, 2024-05-27T08:48:21.834 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-05-27T08:48:21.834 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.834 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-05-27T08:48:21.834 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.834 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.834 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.835 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:21.835 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:21.835 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.835 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.835 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.835 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:21.835 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.835 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.836 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.836 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:48:21.836 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.836 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.836 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.836 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.836 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.836 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.837 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.838 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.838 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.838 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.838 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.838 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.838 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.838 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.838 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.839 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.839 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.839 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.839 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.839 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.839 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.839 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.839 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.840 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.841 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.841 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.841 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.841 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.841 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.841 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.841 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.841 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.842 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.842 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.842 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.842 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.842 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.842 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.842 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.842 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.843 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.843 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.843 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.843 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.843 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.843 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.843 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.843 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.844 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.845 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.845 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-05-27T08:48:21.845 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-05-27T08:48:21.845 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.845 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.845 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.845 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.845 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.846 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.846 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.846 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:48:21.846 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.846 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.846 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.846 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.846 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-05-27T08:48:21.847 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-05-27T08:48:21.847 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.847 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.847 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.847 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-05-27T08:48:21.847 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.847 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.847 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.848 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.849 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.849 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.849 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.849 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.849 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.849 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-05-27T08:48:21.849 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.849 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.850 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.850 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-05-27T08:48:21.850 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-05-27T08:48:21.850 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.850 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.850 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.851 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.852 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-05-27T08:48:21.852 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-05-27T08:48:21.852 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.852 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.852 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.852 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-05-27T08:48:21.852 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.852 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-05-27T08:48:21.853 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.854 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.854 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.854 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.854 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.854 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.854 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.854 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:48:21.854 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.855 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.855 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.855 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.855 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-05-27T08:48:21.855 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-05-27T08:48:21.855 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.855 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.855 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.856 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.856 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.856 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.856 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.856 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:48:21.856 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.856 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.856 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.857 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.858 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.858 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-05-27T08:48:21.858 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.858 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.858 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.858 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.858 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.858 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.859 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.859 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.859 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.859 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.859 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.859 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.859 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.859 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.860 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.860 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.860 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.860 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.860 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.860 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.860 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.860 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.861 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.862 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.862 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.862 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.862 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:48:21.862 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.862 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.862 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.862 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.863 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-05-27T08:48:21.863 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-05-27T08:48:21.863 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.863 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.863 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.863 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-05-27T08:48:21.863 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.863 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-05-27T08:48:21.864 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.865 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.865 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.865 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.865 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-05-27T08:48:21.865 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-05-27T08:48:21.865 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.865 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:48:21.865 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:48:21.866 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.866 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.866 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-05-27T08:48:21.866 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:48:21.866 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.866 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.866 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.867 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.868 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-05-27T08:48:21.868 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-05-27T08:48:21.868 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.868 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.868 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.868 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:48:21.868 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.868 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-05-27T08:48:21.869 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.870 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.870 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.870 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-05-27T08:48:21.870 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-05-27T08:48:21.870 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-05-27T08:48:21.870 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.870 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:48:21.870 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:48:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-05-27T08:48:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-05-27T08:48:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.871 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.872 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-05-27T08:48:21.872 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.872 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.872 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.872 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:48:21.872 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:48:21.872 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.873 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-05-27T08:48:21.874 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-05-27T08:48:21.874 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.874 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:48:21.874 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:48:21.874 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.874 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.874 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.874 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.875 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.875 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.875 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.875 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.875 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.875 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.875 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.875 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.876 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.876 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.876 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.876 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.876 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.876 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.876 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.876 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.877 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.877 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.877 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.877 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.877 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.877 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.877 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.877 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.878 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.879 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.879 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.879 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.879 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.879 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.879 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.879 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.879 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.880 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.880 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.880 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.880 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.880 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.880 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.880 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.880 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.881 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.882 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.882 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.882 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.882 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.882 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.882 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.882 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.882 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.883 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.883 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.883 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.883 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-05-27T08:48:21.883 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-05-27T08:48:21.883 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.883 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.883 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.884 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.884 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.884 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.884 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.884 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:48:21.884 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.884 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.884 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.885 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.886 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.886 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-05-27T08:48:21.886 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-05-27T08:48:21.886 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-05-27T08:48:21.886 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-05-27T08:48:21.886 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-05-27T08:48:21.886 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.886 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-05-27T08:48:21.887 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.887 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.887 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.887 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.887 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-05-27T08:48:21.887 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-05-27T08:48:21.887 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.887 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.888 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.888 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.888 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.888 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.888 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.888 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:48:21.888 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.888 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:21.889 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.890 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.890 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.890 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:48:21.890 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.890 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.890 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.890 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.890 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.891 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.891 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.891 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.891 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.891 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.891 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.891 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.891 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.892 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.892 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.892 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.892 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.892 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-05-27T08:48:21.892 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-05-27T08:48:21.892 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:48:21.893 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.894 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.894 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.894 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-05-27T08:48:21.894 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:48:21.894 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:21.894 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.894 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.895 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.896 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:21.896 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.896 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:21.896 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-05-27T08:48:21.896 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:21.896 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:21.896 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:21.896 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-05-27T08:48:21.897 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-05-27T08:48:21.897 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.897 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.897 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.897 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-05-27T08:48:21.897 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.897 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.897 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.898 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.899 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.899 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-05-27T08:48:21.899 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.899 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.899 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.899 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:48:21.899 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.899 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-05-27T08:48:21.900 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.901 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.901 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.901 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:48:21.901 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.901 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.901 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.901 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.901 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-05-27T08:48:21.902 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-05-27T08:48:21.902 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.902 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.902 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.902 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.902 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.902 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.902 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.903 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:48:21.903 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:48:21.903 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.903 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.903 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.903 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-05-27T08:48:21.903 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-05-27T08:48:21.903 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.904 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.904 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.904 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:21.904 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.904 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.904 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.904 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:48:21.904 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.905 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-05-27T08:48:21.906 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.906 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.906 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.906 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:48:21.906 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.906 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.906 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.906 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.907 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-05-27T08:48:21.907 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-05-27T08:48:21.907 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.907 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.907 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.907 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.907 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.907 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.908 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.908 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-05-27T08:48:21.908 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-05-27T08:48:21.908 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-05-27T08:48:21.908 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.908 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:48:21.908 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.908 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.909 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.909 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.909 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-05-27T08:48:21.909 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-05-27T08:48:21.909 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.909 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.909 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.909 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.910 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-05-27T08:48:21.911 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-05-27T08:48:21.911 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.911 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.911 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.911 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.911 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.911 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.911 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:48:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-05-27T08:48:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-05-27T08:48:21.912 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.913 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.914 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.914 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.914 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-05-27T08:48:21.914 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-05-27T08:48:21.914 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.914 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.914 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.915 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:48:21.915 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.915 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.915 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.915 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:21.915 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.915 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.915 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.916 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.916 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-05-27T08:48:21.916 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-05-27T08:48:21.916 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.916 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.916 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.916 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:48:21.916 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.917 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.917 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.917 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:48:21.917 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.917 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.917 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.917 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.917 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-05-27T08:48:21.918 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:48:21.918 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.918 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.918 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.918 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:48:21.918 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.918 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.918 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-05-27T08:48:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-05-27T08:48:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-05-27T08:48:21.919 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.920 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.920 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.920 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:48:21.920 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.920 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.920 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.920 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:21.920 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.921 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:48:21.922 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.922 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.922 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.922 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-05-27T08:48:21.922 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.922 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.922 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.922 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.923 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-05-27T08:48:21.923 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:48:21.923 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.923 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.923 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.923 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:48:21.923 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.923 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.924 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.924 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:48:21.924 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.924 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.924 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.924 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.924 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-05-27T08:48:21.924 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:48:21.925 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.925 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.925 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.925 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:48:21.925 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.925 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.925 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.925 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:48:21.926 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.926 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.926 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.926 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.926 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-05-27T08:48:21.926 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:48:21.926 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.926 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.927 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.927 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:48:21.927 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.927 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.927 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.927 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-05-27T08:48:21.927 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.927 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.928 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.928 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.928 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-05-27T08:48:21.928 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-05-27T08:48:21.928 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.928 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.928 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.928 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.929 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-05-27T08:48:21.930 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-05-27T08:48:21.930 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.930 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.930 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.930 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:48:21.930 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.930 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.930 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.931 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-05-27T08:48:21.931 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.931 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.931 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.931 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.931 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-05-27T08:48:21.931 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-05-27T08:48:21.931 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.932 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.933 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.933 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.933 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-05-27T08:48:21.933 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-05-27T08:48:21.933 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.933 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.933 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.933 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:48:21.934 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.934 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.934 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.934 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-05-27T08:48:21.934 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.934 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.934 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.934 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.935 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-05-27T08:48:21.935 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-05-27T08:48:21.935 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.935 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.935 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.935 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:48:21.935 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.935 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.936 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.936 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:21.936 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.936 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.936 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.936 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.936 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-05-27T08:48:21.936 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-05-27T08:48:21.937 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.937 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.937 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.937 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:48:21.937 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.937 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.937 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.937 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-05-27T08:48:21.938 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.938 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.938 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.938 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.938 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-05-27T08:48:21.938 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-05-27T08:48:21.938 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.938 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.939 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.940 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.940 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-05-27T08:48:21.940 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:48:21.940 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.940 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.940 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.940 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:48:21.940 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.941 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.941 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.941 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:48:21.941 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.941 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.941 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.941 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.941 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-05-27T08:48:21.942 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-05-27T08:48:21.942 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.942 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.942 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.942 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.942 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.942 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.942 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.943 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.944 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.944 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:21.944 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.944 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.944 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.944 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:48:21.944 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.944 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-05-27T08:48:21.945 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.946 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.946 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.946 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:48:21.946 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.946 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.946 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.946 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.946 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-05-27T08:48:21.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-05-27T08:48:21.947 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.947 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.947 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.947 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-05-27T08:48:21.947 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.947 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.947 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.948 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:48:21.948 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.948 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.948 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.948 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.948 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-05-27T08:48:21.948 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-05-27T08:48:21.948 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.949 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.949 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.949 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-05-27T08:48:21.949 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.949 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.949 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.949 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:48:21.949 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.950 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.950 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.950 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.950 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-05-27T08:48:21.950 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-05-27T08:48:21.950 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.950 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.950 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.951 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.951 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.951 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.951 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.951 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:48:21.951 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.951 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.951 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.952 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-05-27T08:48:21.952 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-05-27T08:48:21.952 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.952 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.952 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.952 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:21.952 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-05-27T08:48:21.953 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:48:21.954 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.954 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.954 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.954 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:48:21.954 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.954 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.954 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.958 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:48:21.958 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.958 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.958 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.958 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-05-27T08:48:21.958 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-05-27T08:48:21.959 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:21.959 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.959 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.959 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:21.959 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.959 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.959 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.960 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:48:21.960 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.960 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.960 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.960 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.960 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-05-27T08:48:21.960 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-05-27T08:48:21.960 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.961 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.961 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.961 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.961 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.961 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.961 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.961 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:48:21.961 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.962 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.963 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.963 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.963 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.963 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:48:21.963 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.963 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.963 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.964 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:21.964 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:21.964 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.964 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.964 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.964 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.964 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.964 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.965 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.966 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.966 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.966 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.966 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-05-27T08:48:21.966 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-05-27T08:48:21.966 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.966 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.967 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.967 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.967 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.967 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.967 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.967 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:48:21.967 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.967 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.968 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.969 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.970 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.971 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.972 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.973 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:48:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:48:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.974 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:48:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.975 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:48:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:48:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-05-27T08:48:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.976 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.977 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:48:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.978 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-05-27T08:48:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-05-27T08:48:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.979 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:48:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.980 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-05-27T08:48:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-05-27T08:48:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.981 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-05-27T08:48:21.982 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-05-27T08:48:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-05-27T08:48:21.983 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-05-27T08:48:21.984 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:21.984 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:48:21.984 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.984 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.984 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.984 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.984 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-05-27T08:48:21.984 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-05-27T08:48:21.985 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.985 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.985 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.985 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-05-27T08:48:21.985 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.985 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.985 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.985 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:48:21.986 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.986 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.986 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.986 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.986 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-05-27T08:48:21.986 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:48:21.986 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.986 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.987 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.988 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.988 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-05-27T08:48:21.988 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-05-27T08:48:21.988 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.988 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.988 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.988 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.988 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.989 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.989 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.989 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:48:21.989 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.989 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.989 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.989 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.989 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-05-27T08:48:21.990 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-05-27T08:48:21.990 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.990 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.990 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.990 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.990 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.990 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.990 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.991 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:48:21.991 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.991 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.991 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.991 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-05-27T08:48:21.991 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-05-27T08:48:21.991 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.992 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.992 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.992 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.992 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.992 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.992 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.992 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:48:21.992 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.993 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.993 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.993 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-05-27T08:48:21.993 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-05-27T08:48:21.993 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:21.993 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.993 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.994 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:21.994 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.994 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.994 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.994 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:48:21.994 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.994 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.994 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.995 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.995 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-05-27T08:48:21.995 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:48:21.995 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:21.995 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.995 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.995 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:21.995 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.996 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.996 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.996 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:48:21.996 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.996 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.996 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.996 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.996 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.997 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:48:21.998 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.998 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:21.998 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:21.998 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:21.998 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:21.998 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:21.998 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:21.999 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.000 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.000 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.000 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-05-27T08:48:22.000 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-05-27T08:48:22.000 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.000 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.000 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.000 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.001 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-05-27T08:48:22.002 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-05-27T08:48:22.002 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.002 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.002 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.002 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.002 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-05-27T08:48:22.003 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-05-27T08:48:22.004 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.004 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.004 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.004 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.004 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.004 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.004 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.004 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:48:22.005 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.005 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.005 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.005 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.005 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-05-27T08:48:22.005 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:48:22.005 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.005 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.006 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.006 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.006 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.006 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.006 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.006 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:48:22.006 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.006 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.007 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.007 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.007 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-05-27T08:48:22.007 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:48:22.007 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.007 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.007 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.007 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.008 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-05-27T08:48:22.009 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:48:22.009 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.009 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.009 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.009 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.009 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.009 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.009 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.010 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:48:22.010 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.010 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.010 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.010 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.010 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.010 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.010 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.011 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-05-27T08:48:22.011 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.011 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.011 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.011 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.011 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.011 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.011 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.012 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.012 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.012 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.012 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.012 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.012 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.012 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.012 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.013 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.014 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.014 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.014 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.014 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.014 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.014 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.014 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.015 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.015 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.015 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.015 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.015 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.015 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.015 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.015 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.016 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.017 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.017 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.017 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.017 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.017 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.017 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.017 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.017 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.018 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.018 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.018 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.018 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.018 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.018 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.018 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.018 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.019 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.020 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.021 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.021 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:48:22.021 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.021 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.021 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.021 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.021 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.021 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.022 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.022 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.022 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.022 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.022 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.022 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.022 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.022 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.023 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.023 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.023 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.023 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.023 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-05-27T08:48:22.023 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-05-27T08:48:22.023 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.023 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.024 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.024 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-05-27T08:48:22.024 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.024 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.024 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.024 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:48:22.024 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.024 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.025 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.025 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.025 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.025 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.025 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.025 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-05-27T08:48:22.025 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.025 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.026 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.026 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:48:22.026 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:48:22.026 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.026 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.026 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.026 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.026 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.027 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.027 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.027 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.027 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.027 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.027 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.027 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.027 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:48:22.028 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:48:22.028 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.028 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.028 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.028 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.028 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.028 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.028 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.029 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.029 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.029 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.029 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.029 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.029 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:48:22.029 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:48:22.029 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.030 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.030 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.030 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.030 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.030 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.030 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.030 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.030 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.031 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.031 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.031 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.031 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:48:22.031 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:48:22.031 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.031 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.031 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.033 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:48:22.033 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:48:22.033 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.033 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.033 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.033 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.033 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.033 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.034 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.034 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.034 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.034 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.034 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.034 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.034 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:48:22.034 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:48:22.035 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.035 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.035 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.035 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.035 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.035 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.035 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.035 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.036 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.036 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.036 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.036 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.036 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:48:22.036 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:48:22.036 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.036 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.037 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.037 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.037 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.037 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.037 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.037 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.037 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.037 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.038 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.038 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.038 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:48:22.038 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:48:22.038 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.038 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.038 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.038 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.039 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.039 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.039 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.039 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.039 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.039 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.039 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.039 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.040 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.041 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.041 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.041 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.041 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.041 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.041 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:48:22.041 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:48:22.041 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.042 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:48:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:48:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.043 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.044 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.045 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:48:22.045 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:48:22.045 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.045 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.045 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.045 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.045 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.045 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.046 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.046 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.046 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.046 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.046 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.046 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.046 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:48:22.046 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:48:22.047 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.047 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.047 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.047 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.047 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.047 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.047 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.047 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.048 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.048 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.048 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.048 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.048 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:48:22.048 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:48:22.048 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.048 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.049 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.049 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.049 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.049 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.049 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.049 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.049 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.049 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.050 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.050 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.050 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:48:22.050 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:48:22.050 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.050 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.050 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.050 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.051 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:48:22.052 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:48:22.052 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.052 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.052 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.052 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.052 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.052 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.052 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.053 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.053 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.053 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.053 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.053 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.053 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-05-27T08:48:22.053 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:48:22.053 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.054 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.054 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.054 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.054 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.054 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.054 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.054 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.054 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.055 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.055 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.055 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.055 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:48:22.055 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:48:22.055 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.055 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.055 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.056 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:48:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:48:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.057 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:48:22.058 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.059 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.060 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.060 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.060 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.060 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:48:22.060 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:48:22.060 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.060 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.060 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.061 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.061 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.061 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.061 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.061 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.061 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.061 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.062 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.062 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.062 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:48:22.062 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:48:22.062 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.062 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.062 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.062 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.063 INFO:teuthology.orchestra.run.smithi077.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "[]", 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.064 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.065 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.065 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.065 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.065 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.065 INFO:teuthology.orchestra.run.smithi077.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:48:22.065 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:48:22.065 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.065 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.066 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.066 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "{}", 2024-05-27T08:48:22.066 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.066 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.066 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.066 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.066 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.066 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.067 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.067 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.067 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:48:22.067 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:48:22.067 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.067 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.067 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.067 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.068 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.068 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.068 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.068 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.068 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.068 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.068 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.068 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.069 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:48:22.070 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.071 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.072 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.072 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.072 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.072 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:48:22.072 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:48:22.072 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.072 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.072 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.073 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.073 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.073 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.073 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.073 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.073 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.073 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.073 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.074 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.074 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:48:22.074 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:48:22.074 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.074 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.074 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.074 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.074 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.075 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.075 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.075 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.075 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.075 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.075 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.075 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.075 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.076 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.077 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.077 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.077 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.077 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.077 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:48:22.077 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:48:22.077 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.077 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.078 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.078 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.078 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.078 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.078 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.078 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.078 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.078 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.079 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.079 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.079 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:48:22.079 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:48:22.079 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.079 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.079 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.079 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.080 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.080 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.080 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.080 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.080 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.080 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.080 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.080 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.081 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:48:22.081 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:48:22.081 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.081 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.081 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.081 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.081 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.081 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.082 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.082 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.082 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.082 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.082 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.082 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.082 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:48:22.082 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:48:22.083 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.083 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.083 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.083 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.083 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.083 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.083 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.083 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.084 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.084 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.084 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.084 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.084 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:48:22.084 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:48:22.084 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.084 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.085 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.086 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.086 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:48:22.086 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:48:22.086 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.086 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.086 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.086 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.086 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.087 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.087 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.087 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.087 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.087 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.087 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.087 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.087 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.088 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.089 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.089 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.089 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.089 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.089 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:48:22.089 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:48:22.089 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.089 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.090 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.090 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-05-27T08:48:22.090 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.090 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.090 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.090 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.090 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.090 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.091 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.091 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.091 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:48:22.091 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:48:22.091 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.091 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.091 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.092 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.092 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.092 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.092 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.092 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.092 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.092 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:22.093 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.094 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.094 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.094 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.094 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.094 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.094 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.094 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.094 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:48:22.095 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:48:22.095 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.095 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.095 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.095 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.095 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.095 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.095 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.096 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.096 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.096 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.096 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.096 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.096 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:48:22.096 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:48:22.096 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.097 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.097 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.097 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.097 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.097 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.097 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.097 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.097 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.098 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.098 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.098 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.098 INFO:teuthology.orchestra.run.smithi077.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:48:22.098 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:48:22.098 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.098 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.098 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.099 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.099 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.099 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.099 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.099 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.099 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.099 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.099 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:48:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:48:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.100 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.101 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.101 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.101 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.101 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.101 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.101 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.101 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.101 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:48:22.102 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:48:22.102 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.102 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.102 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.102 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.102 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.102 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.102 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.103 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.104 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.104 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:22.104 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.104 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.104 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.104 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.104 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.104 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-05-27T08:48:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-05-27T08:48:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.105 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.106 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.106 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.106 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.106 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.106 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.106 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.106 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.106 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.107 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.108 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.108 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.108 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.108 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.108 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.108 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-05-27T08:48:22.108 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-05-27T08:48:22.108 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.109 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.109 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.109 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.109 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.109 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.109 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.109 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:48:22.109 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.110 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-05-27T08:48:22.111 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.111 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.111 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.111 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.111 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.111 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.111 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.111 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.112 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-05-27T08:48:22.112 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-05-27T08:48:22.112 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.112 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.112 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.112 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.112 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.112 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.113 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.114 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.114 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.114 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.114 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.114 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.114 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.114 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.114 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.115 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.115 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.115 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.115 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.115 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.115 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.115 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.116 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.117 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.117 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.117 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.117 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.117 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.117 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.117 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.117 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.118 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.118 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.118 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.118 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.118 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.118 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.118 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.118 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.119 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.119 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.119 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.119 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.119 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.119 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.119 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.119 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.120 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.120 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.120 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.120 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.120 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.120 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.120 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.120 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.121 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.122 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.122 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-05-27T08:48:22.122 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-05-27T08:48:22.122 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.122 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.122 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.123 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.124 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:48:22.124 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:48:22.124 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.124 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.124 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.124 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.124 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.124 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.125 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.125 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.125 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.125 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.125 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.125 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:22.125 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-05-27T08:48:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.126 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.127 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.128 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.129 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.129 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.129 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.129 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.129 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.129 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.129 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.129 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.130 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.130 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.130 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.130 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.130 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.130 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.130 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.130 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.131 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.132 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.132 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.132 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.132 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-05-27T08:48:22.132 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-05-27T08:48:22.132 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.132 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.133 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.133 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-05-27T08:48:22.133 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.133 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.133 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.133 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.133 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.133 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.134 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.134 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-05-27T08:48:22.134 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-05-27T08:48:22.134 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.134 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.134 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.134 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.135 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.136 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.136 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-05-27T08:48:22.136 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-05-27T08:48:22.136 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.136 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.136 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.136 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-05-27T08:48:22.137 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-05-27T08:48:22.137 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-05-27T08:48:22.137 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.137 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.137 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.137 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.137 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.137 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.138 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-05-27T08:48:22.138 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-05-27T08:48:22.138 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.138 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.138 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.138 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.138 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.139 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.139 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.139 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.139 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.139 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.139 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.139 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.139 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.140 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.140 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.140 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-05-27T08:48:22.140 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.140 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.140 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.140 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-05-27T08:48:22.140 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-05-27T08:48:22.141 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.142 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.142 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.142 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.142 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.142 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.142 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.142 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.143 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.143 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.143 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.143 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.143 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.143 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.143 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.143 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.145 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.145 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.145 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.145 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.145 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.145 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.145 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.145 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.146 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.146 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.146 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.146 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.146 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.146 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.146 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.146 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.147 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.147 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.147 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.147 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.147 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.147 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.147 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.147 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.148 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.148 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.148 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.148 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.148 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.148 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.148 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.148 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.149 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.149 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.149 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.149 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.149 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.149 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.149 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.150 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.150 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.150 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.150 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.150 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.150 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.150 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.150 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.151 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.151 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-05-27T08:48:22.151 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-05-27T08:48:22.151 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.151 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.151 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.151 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-05-27T08:48:22.151 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.152 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.152 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.152 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:48:22.152 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.152 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.152 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.152 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.153 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:48:22.154 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.154 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.154 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.154 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-05-27T08:48:22.154 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-05-27T08:48:22.154 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.154 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.155 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.155 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-05-27T08:48:22.155 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.155 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.155 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.155 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:48:22.155 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.155 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.156 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.156 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-05-27T08:48:22.156 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-05-27T08:48:22.156 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.156 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.156 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.156 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-05-27T08:48:22.157 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.157 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.157 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.157 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:48:22.157 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.157 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.157 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.157 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.158 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.159 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:48:22.159 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.159 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.159 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.162 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:22.162 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:22.162 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.162 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.162 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.163 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:22.163 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.163 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.163 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.163 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:48:22.163 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.163 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.163 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.164 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.164 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.164 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.164 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.164 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.164 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.165 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.165 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.165 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.165 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.165 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.165 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.165 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.165 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.166 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-05-27T08:48:22.166 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-05-27T08:48:22.166 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.166 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.166 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.166 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-05-27T08:48:22.166 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.168 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.168 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-05-27T08:48:22.168 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.168 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.168 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.168 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.168 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.168 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.169 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.169 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.169 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.169 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.169 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.169 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.169 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.169 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.170 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.170 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.170 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.170 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.170 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.170 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.170 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.170 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.171 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.172 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.172 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.172 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.172 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.172 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.172 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.172 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.172 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.173 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.173 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.173 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.173 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.173 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.173 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.173 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.173 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.174 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.174 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.174 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.174 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.174 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.174 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.174 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.174 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.175 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.175 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.175 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.175 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.175 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.175 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.175 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.176 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.176 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.176 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.176 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.176 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.176 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.176 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.176 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-05-27T08:48:22.177 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.178 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.178 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.178 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.178 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.178 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.178 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.178 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.179 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.180 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.180 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.180 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.180 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.180 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:22.180 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:22.180 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.180 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.181 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.181 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:22.181 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.181 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.181 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.181 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.181 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.181 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.182 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.183 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.183 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.183 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.183 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.183 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.183 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.184 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.184 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.184 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-05-27T08:48:22.184 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-05-27T08:48:22.184 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:48:22.184 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.184 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-05-27T08:48:22.184 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:48:22.185 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.186 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.186 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.186 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.186 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-05-27T08:48:22.186 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-05-27T08:48:22.186 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.186 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.186 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.187 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-05-27T08:48:22.187 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.187 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.187 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.187 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:48:22.187 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.187 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.187 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.188 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.188 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-05-27T08:48:22.188 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-05-27T08:48:22.188 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.188 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.188 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.188 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.188 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.189 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.189 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.189 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:48:22.189 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.189 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.189 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.189 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.189 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:22.190 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:22.190 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.190 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.190 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.190 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-05-27T08:48:22.190 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-05-27T08:48:22.190 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.190 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.191 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:48:22.191 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.191 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.191 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.191 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.191 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.191 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.191 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.192 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.193 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.193 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.193 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.193 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.193 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.193 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.193 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.193 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.194 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.194 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.194 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.194 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.194 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.194 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.194 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.194 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.195 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.195 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.195 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.195 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.195 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.195 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.195 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.195 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.196 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.196 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.196 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.196 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.196 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.196 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.196 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.196 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.197 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.198 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.198 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.198 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.198 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.198 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.198 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.198 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.198 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.199 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.199 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.199 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.199 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.199 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.199 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.199 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.199 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.200 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.200 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-05-27T08:48:22.200 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-05-27T08:48:22.200 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.200 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.200 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.200 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.200 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-05-27T08:48:22.201 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-05-27T08:48:22.202 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.202 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.202 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.202 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-05-27T08:48:22.202 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.202 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.202 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.203 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-05-27T08:48:22.203 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.203 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.203 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.203 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.203 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.203 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.203 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.204 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.205 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.205 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.205 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-05-27T08:48:22.205 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-05-27T08:48:22.205 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.205 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.205 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.205 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-05-27T08:48:22.206 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.206 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.206 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.206 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:48:22.206 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.206 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.206 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.206 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-05-27T08:48:22.207 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.208 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:48:22.208 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.208 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.208 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.208 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.208 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-05-27T08:48:22.208 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-05-27T08:48:22.208 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.209 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.209 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.209 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.209 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.209 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.209 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.209 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:48:22.209 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.210 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.210 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.210 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.210 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-05-27T08:48:22.210 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-05-27T08:48:22.210 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.210 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.210 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.211 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-05-27T08:48:22.211 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.211 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.211 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.211 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:48:22.211 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.211 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.211 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.212 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.212 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.212 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.212 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.212 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-05-27T08:48:22.212 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.212 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.212 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.213 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.214 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.214 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.214 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.214 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.214 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.214 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.214 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.214 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.215 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.215 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.215 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.215 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.215 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.215 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.215 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.215 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.216 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.217 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.217 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.217 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.217 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.217 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.217 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.217 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.217 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.218 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.218 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.218 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.218 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.218 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.218 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.218 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.218 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.219 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.220 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.221 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.221 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.221 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.221 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.221 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.221 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.221 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.221 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.222 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.222 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.222 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.222 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.222 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.222 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.222 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.222 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.223 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.223 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.223 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.223 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.223 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-05-27T08:48:22.223 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.223 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.223 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.224 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.224 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.224 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.224 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.224 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.224 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.224 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.224 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.225 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.225 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.225 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.225 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.225 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.225 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.225 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.225 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.226 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.226 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.226 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.226 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.226 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.226 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.226 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.226 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.227 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.228 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.228 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.228 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.228 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.228 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.228 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.228 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.228 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.229 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.229 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.229 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.229 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.229 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.229 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.229 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.229 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.230 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.230 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.230 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.230 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.230 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.230 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.230 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.230 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.231 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.231 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.231 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.231 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.231 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.231 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.231 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.231 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.232 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.233 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.233 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.233 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.233 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.233 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.233 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.233 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.233 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.234 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.234 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.234 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.234 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.234 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.234 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-05-27T08:48:22.234 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.234 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.235 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.235 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-05-27T08:48:22.235 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:48:22.235 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.235 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.235 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.235 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-05-27T08:48:22.235 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-05-27T08:48:22.236 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-05-27T08:48:22.237 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.237 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.237 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.237 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.237 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-05-27T08:48:22.237 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.237 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.237 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:48:22.238 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.238 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.238 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.238 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.238 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.238 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.238 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.238 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.239 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.239 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.239 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.239 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.239 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.239 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.239 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.239 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.240 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.240 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.240 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.240 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.240 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.240 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.240 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.240 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.241 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.242 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.242 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.242 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.242 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.242 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.242 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.242 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.242 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.243 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.243 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.243 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.243 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.243 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.243 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.243 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.244 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.245 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.245 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.245 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.245 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.245 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.245 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.245 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.245 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.246 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.247 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.247 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.247 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.247 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.247 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.247 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.247 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.247 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.248 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.248 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.248 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.248 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.248 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.248 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.248 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.248 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.249 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.249 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-05-27T08:48:22.249 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.249 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.249 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.249 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-05-27T08:48:22.249 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-05-27T08:48:22.249 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.250 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.250 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.250 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-05-27T08:48:22.250 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.250 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.250 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.250 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:48:22.250 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.251 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.252 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.252 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.252 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.252 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.252 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.252 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.252 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.252 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.253 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.254 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.254 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.254 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.254 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.254 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.254 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.254 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.254 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.255 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.256 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.256 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.256 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.256 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.256 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.256 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.256 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.256 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.257 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.258 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.258 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.258 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.258 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.258 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.258 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.258 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.258 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.259 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.259 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.259 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.259 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.259 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.259 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.259 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.259 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.260 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:48:22.261 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:48:22.261 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.261 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.261 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.261 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-05-27T08:48:22.261 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-05-27T08:48:22.261 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.261 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.262 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.262 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-05-27T08:48:22.262 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.262 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.262 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.262 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:48:22.262 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.262 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.263 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.263 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.263 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-05-27T08:48:22.263 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-05-27T08:48:22.263 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.263 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.263 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.263 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-05-27T08:48:22.264 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.264 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.264 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.264 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:48:22.264 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.264 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.264 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.264 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.265 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:48:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.266 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.267 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.267 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.267 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.267 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.267 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.267 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.267 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.267 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.268 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.268 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.268 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.268 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-05-27T08:48:22.268 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-05-27T08:48:22.268 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.268 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.268 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.269 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-05-27T08:48:22.269 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.269 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.269 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.269 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:48:22.269 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:48:22.269 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.269 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.270 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.270 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.270 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.270 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.270 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-05-27T08:48:22.270 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.270 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.270 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.271 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.271 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.271 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.271 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.271 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.271 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.271 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.271 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.272 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.272 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.272 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.272 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.272 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.272 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.272 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.272 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.273 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.273 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.273 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.273 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.273 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.273 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.273 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.273 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.274 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.275 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.275 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.275 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.275 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.275 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.275 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.275 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.275 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.276 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.276 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.276 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.276 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.276 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.276 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.276 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.276 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.277 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.277 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.277 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.277 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.277 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.277 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.277 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.277 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.278 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.279 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.279 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.279 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.279 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.279 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.279 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.279 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.279 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.280 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.280 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.280 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.280 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.280 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.280 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.280 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.280 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.281 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.281 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.281 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.281 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.281 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.281 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-05-27T08:48:22.281 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.281 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.282 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.282 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.282 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.282 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.282 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.282 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.282 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.282 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.283 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.283 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.283 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.283 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.283 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.283 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.283 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.283 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.284 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.284 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.284 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.284 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.284 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.284 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.284 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.284 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.285 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.285 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.285 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.285 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.285 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.285 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.285 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.285 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.286 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.286 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.286 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.286 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.286 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.286 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.286 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.287 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.287 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.287 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.287 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.287 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.287 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.287 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.288 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.289 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.289 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.289 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.289 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.289 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.289 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.289 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.289 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.290 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.290 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.290 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.290 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.290 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.290 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.290 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.291 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.292 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.292 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.292 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.292 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.292 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.292 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.292 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.292 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.293 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.294 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.294 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.294 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.294 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.294 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.294 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.294 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.295 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.295 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.295 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.295 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.295 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.295 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.295 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.295 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.296 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.296 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.296 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.296 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.296 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.296 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.296 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.296 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.297 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.297 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.297 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.297 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.297 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.297 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.297 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.297 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.298 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.298 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.298 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.298 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.298 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.298 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.298 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.298 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.299 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.299 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.299 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.299 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.299 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.299 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.299 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.299 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.300 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.300 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.300 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.300 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.300 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.300 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.300 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.301 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.302 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.302 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.302 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.302 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.302 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.302 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.302 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.302 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.303 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-05-27T08:48:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-05-27T08:48:22.304 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-05-27T08:48:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.305 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:48:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.306 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.307 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.308 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.309 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.309 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.309 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.309 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.309 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.309 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.309 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.309 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.310 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.310 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.310 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.310 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.310 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.310 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.310 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.311 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.312 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.312 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.312 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.312 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.312 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.312 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.312 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.312 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.313 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.314 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.314 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.314 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.314 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.314 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.314 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.314 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.315 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.315 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-05-27T08:48:22.315 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-05-27T08:48:22.315 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.315 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.315 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.315 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.315 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.316 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.317 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.317 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.317 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.317 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.317 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.317 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.317 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.317 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.318 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.318 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.318 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.318 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.318 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.318 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.318 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.318 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.319 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.319 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.319 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.319 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-05-27T08:48:22.319 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.319 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.319 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.319 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.320 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.320 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.320 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.320 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.320 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.320 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.320 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.320 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.321 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.321 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.321 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.321 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.321 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.321 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.321 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.321 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.322 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.322 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.322 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.322 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.322 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.322 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.322 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.323 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.323 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.323 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.323 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.323 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.323 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.323 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.323 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.324 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.325 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.325 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.325 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.325 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.325 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.325 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.325 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.325 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.326 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.326 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.326 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.326 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.326 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.326 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.326 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.326 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.327 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.327 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.327 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.327 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.327 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.327 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.327 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.327 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.328 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.328 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.328 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.328 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.328 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.328 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.328 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.328 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.329 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.330 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.330 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.330 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.330 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.330 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-05-27T08:48:22.330 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.330 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.330 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.331 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.331 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.331 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.331 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.331 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.331 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.331 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.331 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.332 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.332 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.332 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.332 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.332 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.332 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.332 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.332 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.333 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.334 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.334 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.334 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.334 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.334 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.334 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.334 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.335 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.335 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.335 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.335 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.335 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.335 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.335 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.335 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.336 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.337 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.337 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.337 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.337 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.337 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.337 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.337 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.337 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.338 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.338 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.338 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.338 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.338 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.338 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.338 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.338 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.339 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.339 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.339 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.339 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.339 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.339 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.339 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.339 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.340 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.341 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.341 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.341 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.341 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.341 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.341 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-05-27T08:48:22.341 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.341 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.342 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.342 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.342 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.342 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.342 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.342 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.342 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.342 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.343 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.343 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.343 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.343 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.343 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.343 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.343 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.343 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.344 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.344 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.344 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.344 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.344 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.344 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.344 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.344 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.345 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.346 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.346 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.346 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.346 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.346 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.346 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.346 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.346 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.347 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.347 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.347 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.347 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.347 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.347 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.347 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.347 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.348 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.348 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.348 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.348 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.348 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.348 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.348 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.348 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.349 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.349 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.349 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.349 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.349 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.349 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.349 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.349 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.350 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.350 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.350 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.350 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.350 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.350 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.350 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:22.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.351 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.352 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.352 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.352 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.352 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.352 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.352 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.352 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.352 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.353 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.353 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.353 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.353 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.353 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.353 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.353 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.353 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.354 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.354 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-05-27T08:48:22.354 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-05-27T08:48:22.354 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:22.354 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.354 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.354 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-05-27T08:48:22.354 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-05-27T08:48:22.355 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.355 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.355 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-05-27T08:48:22.355 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:48:22.355 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.355 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.355 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.355 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.356 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.356 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.356 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-05-27T08:48:22.356 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.356 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.356 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.356 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-05-27T08:48:22.356 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:48:22.357 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.357 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.357 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.357 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.357 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.357 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.357 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.357 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:48:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-05-27T08:48:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-05-27T08:48:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.358 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.359 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.360 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.360 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.360 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.360 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.360 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.360 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.360 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.360 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.361 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.362 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.362 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.362 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.362 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.362 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.362 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.362 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.362 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.363 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.363 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.363 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.363 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.363 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.363 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.363 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.363 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.364 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.364 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.364 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.364 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.364 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.364 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.364 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.364 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.365 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.365 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.365 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.365 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.365 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.365 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.365 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.365 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.366 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.367 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.367 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.367 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.367 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.367 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.367 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.367 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.367 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.368 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.368 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.368 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.368 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.368 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.368 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-05-27T08:48:22.368 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-05-27T08:48:22.368 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.369 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.369 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.369 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-05-27T08:48:22.369 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.369 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.369 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.369 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:48:22.369 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.370 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.370 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.370 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.370 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:22.370 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:22.370 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.370 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.371 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.371 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:22.371 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.371 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.371 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.371 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:48:22.371 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.371 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.372 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.373 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.373 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.373 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.373 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.373 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.373 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.373 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.373 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.374 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.374 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.374 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-05-27T08:48:22.374 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.374 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.374 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.374 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-05-27T08:48:22.374 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-05-27T08:48:22.375 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.375 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.375 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.375 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.375 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.375 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.375 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.375 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.376 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.376 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.376 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.376 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.376 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-05-27T08:48:22.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-05-27T08:48:22.376 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.376 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.377 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.377 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.377 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.377 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.377 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.377 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:48:22.377 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:48:22.377 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.378 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.378 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.378 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.378 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.378 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.378 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.378 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.378 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.379 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.380 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.381 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.382 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.383 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.383 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.383 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.383 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.383 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.383 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.383 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.383 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.384 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.385 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.385 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.385 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.385 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.385 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.385 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.385 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.385 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.386 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.386 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.386 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.386 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.386 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.386 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.386 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.387 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:48:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:48:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.388 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.389 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.389 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-05-27T08:48:22.389 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.389 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.389 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.389 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.389 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.389 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.390 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.390 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.390 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-05-27T08:48:22.390 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-05-27T08:48:22.390 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:22.390 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.390 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.390 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-05-27T08:48:22.391 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.391 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.391 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.391 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.391 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.391 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.391 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.391 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.392 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.393 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:48:22.393 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.393 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.393 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.393 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.393 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:22.393 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:22.393 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.394 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.394 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.394 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-05-27T08:48:22.394 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.394 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.394 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.394 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:48:22.394 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.395 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.395 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.395 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.395 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.395 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.395 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.395 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.395 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.396 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.396 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.396 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.396 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.396 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.396 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.396 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.396 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.397 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-05-27T08:48:22.397 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-05-27T08:48:22.397 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.397 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.397 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.397 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-05-27T08:48:22.397 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.398 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.398 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.398 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.398 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.398 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.398 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.398 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.399 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.400 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-05-27T08:48:22.400 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-05-27T08:48:22.400 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.400 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.400 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.400 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.400 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-05-27T08:48:22.401 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-05-27T08:48:22.402 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.402 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.402 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.402 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.402 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.402 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.402 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.402 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.403 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.403 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-05-27T08:48:22.403 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.403 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.403 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.403 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.403 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.403 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.404 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.404 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.404 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.404 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.404 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.404 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.404 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.404 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.405 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.405 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.405 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.405 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.405 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.405 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.405 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.405 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.406 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.406 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.406 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.406 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.406 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.406 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.406 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.406 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.407 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.408 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.409 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.409 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.409 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.409 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.409 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.409 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.409 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.409 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.410 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.410 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.410 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.410 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.410 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.410 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.410 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.410 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.411 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.411 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.411 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.411 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.411 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.411 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.411 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.411 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.412 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.412 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-05-27T08:48:22.412 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:48:22.412 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.412 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.412 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.412 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.412 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:48:22.413 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:48:22.414 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.414 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.414 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.414 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.414 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.414 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.414 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.414 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.415 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.416 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.416 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.416 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.416 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.416 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.416 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.416 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.416 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.417 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.418 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.418 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.418 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.418 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.418 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.418 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.418 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.418 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.419 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.420 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.420 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.420 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.420 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.420 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.420 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.420 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.421 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.422 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.422 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.422 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.422 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.422 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.422 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.422 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.422 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.423 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.423 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.423 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.423 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.423 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.423 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.423 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.423 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.424 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.424 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.424 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.424 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.424 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.424 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.424 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.424 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.425 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.426 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.426 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.426 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.426 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.426 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.426 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.426 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.427 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.428 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.429 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.429 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.429 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.429 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.429 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.429 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.429 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.429 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.430 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.430 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.430 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.430 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.430 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.430 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.430 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.430 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.431 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.432 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.432 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.432 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.432 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.432 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.432 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.432 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.432 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.433 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.434 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.434 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.434 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.434 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.434 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.434 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.434 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.435 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.435 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.435 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.435 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.435 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.435 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.435 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.436 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.437 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-05-27T08:48:22.437 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.437 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.437 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.437 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.437 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.437 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.437 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.438 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.438 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.438 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.438 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.438 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.438 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.438 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.438 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.439 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.439 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.439 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.439 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.439 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.439 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.439 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.439 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.440 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.440 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.440 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.440 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.440 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.440 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.440 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.441 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.442 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.442 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.442 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.442 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.442 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.442 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.442 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.442 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.443 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.443 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.443 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.443 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.443 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.443 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.443 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.443 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.444 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.445 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.445 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.445 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.445 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.445 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.445 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.445 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.446 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.446 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.446 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.446 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.446 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.446 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.446 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.446 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.448 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.448 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-05-27T08:48:22.448 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.448 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.448 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.448 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.448 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.448 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.449 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.449 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.449 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.449 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.449 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.449 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.449 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.449 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.450 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.450 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.450 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.450 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.450 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.450 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.450 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.450 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.451 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.451 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.451 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.451 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.451 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.451 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.451 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.451 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.452 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.452 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.452 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.452 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.452 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.452 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.452 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.452 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.453 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.453 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.453 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.453 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.453 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.453 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.453 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.453 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.454 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.454 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.454 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.454 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.454 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.454 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.454 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.454 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.455 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.456 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.456 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.456 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.456 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.456 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.456 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.456 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.456 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.457 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.457 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.457 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.457 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.457 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.457 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.457 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.457 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.458 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.458 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.458 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.458 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.458 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.458 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.458 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.459 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.460 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.460 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:48:22.460 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.460 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.460 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.460 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.460 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.460 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.461 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.461 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-05-27T08:48:22.461 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.461 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.461 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.461 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.461 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.461 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.462 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.462 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.462 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.462 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.462 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.462 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.462 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.462 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.463 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.463 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.463 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.463 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.463 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.463 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.463 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.463 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.464 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.465 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.465 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.465 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.465 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.465 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.465 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.465 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.465 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.466 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.466 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.466 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.466 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.466 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.466 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.466 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.466 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.467 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.468 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.469 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.469 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.469 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.469 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.469 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.469 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.469 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.469 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.470 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.470 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-05-27T08:48:22.470 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-05-27T08:48:22.470 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.470 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.470 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.470 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.471 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.472 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-05-27T08:48:22.472 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-05-27T08:48:22.472 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.472 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.472 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.472 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-05-27T08:48:22.472 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.472 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.473 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.473 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.473 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.473 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.473 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.473 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.473 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-05-27T08:48:22.473 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-05-27T08:48:22.474 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.474 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.474 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.474 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.474 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.474 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.474 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.474 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.475 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.476 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.476 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.476 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.476 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.476 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.476 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.476 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.476 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.477 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.478 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.478 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.478 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.478 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.478 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.478 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.478 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-05-27T08:48:22.478 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-05-27T08:48:22.479 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.479 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.479 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.479 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.479 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.479 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.479 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.479 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.480 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.480 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.480 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.480 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.480 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-05-27T08:48:22.480 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-05-27T08:48:22.480 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.480 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.481 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.481 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.481 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.481 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.481 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.481 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.481 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.481 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.482 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.482 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.482 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-05-27T08:48:22.482 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-05-27T08:48:22.482 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.482 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.482 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.483 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.484 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-05-27T08:48:22.484 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-05-27T08:48:22.484 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.484 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.484 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.484 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.484 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:22.484 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-05-27T08:48:22.485 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.485 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.485 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.485 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.485 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.485 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.485 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.485 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.486 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.486 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.486 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.486 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.486 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.486 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.486 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.486 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.487 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.487 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.487 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.487 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.487 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-05-27T08:48:22.487 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-05-27T08:48:22.487 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.487 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.488 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.488 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.488 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.488 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.488 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.488 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.488 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.488 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.489 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.489 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.489 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-05-27T08:48:22.489 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-05-27T08:48:22.489 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.489 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.489 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.489 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.490 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.490 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.490 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.490 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.490 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.490 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.490 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.491 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.492 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.492 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.492 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.492 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.492 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.492 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.492 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.492 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.493 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smb", 2024-05-27T08:48:22.493 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.493 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.493 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.493 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.493 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.493 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.493 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.494 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.494 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.494 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.494 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.494 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.494 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.494 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.495 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.496 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.496 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.496 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.496 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.496 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.496 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.496 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.497 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.498 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.498 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.498 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.498 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.498 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.498 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.498 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.499 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.500 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.500 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.500 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.500 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.500 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.500 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.500 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.500 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.501 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.501 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.501 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.501 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.501 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.501 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.501 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.501 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.502 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.502 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.502 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.502 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.502 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.502 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.502 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.502 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.503 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.503 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.503 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.503 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.503 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.503 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.503 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.503 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.504 INFO:teuthology.orchestra.run.smithi077.stdout: "update_orchestration": { 2024-05-27T08:48:22.504 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "update_orchestration", 2024-05-27T08:48:22.504 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.504 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.504 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.504 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.504 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.504 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.505 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.505 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:48:22.505 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.505 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.505 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.505 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.505 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.505 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.506 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.506 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-05-27T08:48:22.506 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.506 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.506 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.506 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-05-27T08:48:22.506 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-05-27T08:48:22.506 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.507 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.507 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.507 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.507 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.507 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.507 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.507 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:48:22.507 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-05-27T08:48:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-05-27T08:48:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.508 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.509 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.510 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.510 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.510 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.510 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.510 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.510 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.510 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.510 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.511 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.512 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.512 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.512 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.512 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.512 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.512 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.512 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.512 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.513 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.514 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.514 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.514 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.514 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.514 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.514 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.514 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.514 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.515 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.515 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.515 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.515 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.515 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.515 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.515 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.515 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.516 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.516 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.516 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.516 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.516 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.516 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.516 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.516 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.517 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.518 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.518 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.518 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.518 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.518 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.518 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.518 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.519 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.520 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.520 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.520 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.520 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.520 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.520 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-05-27T08:48:22.520 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.520 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.521 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.521 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.521 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.521 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.521 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.521 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.521 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.521 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.522 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.522 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.522 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.522 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.522 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.522 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.522 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.522 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.523 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.523 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.523 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.523 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.523 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.523 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.523 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.523 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.524 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.524 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.524 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.524 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.524 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.524 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.524 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.524 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.525 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.525 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.525 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.525 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.525 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.525 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.525 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.526 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.526 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.526 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.526 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.526 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.526 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.526 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.526 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.527 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.527 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.527 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.527 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.527 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.527 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.527 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.527 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.528 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.528 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.528 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.528 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.528 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.528 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.528 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.528 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.529 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.529 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.529 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.529 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.529 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.529 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.529 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.529 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.530 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.530 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.530 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.530 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.530 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.530 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.530 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.530 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.531 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.531 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.531 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.531 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.531 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.531 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.531 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.531 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.532 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-05-27T08:48:22.532 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.532 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.532 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.532 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.532 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.532 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.532 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.533 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.534 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.534 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.534 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.534 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.534 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.534 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.534 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.535 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.535 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.535 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.535 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.535 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.535 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.535 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.535 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.536 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.536 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.536 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.536 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.536 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.536 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.536 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.536 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.537 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.538 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.538 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.538 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.538 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.538 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.538 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.538 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.539 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.540 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.540 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.540 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.540 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.540 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.540 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.540 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.541 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.541 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.541 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.541 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.541 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.541 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.541 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.541 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.542 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.542 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.542 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.542 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.542 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.542 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.542 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.543 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.543 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-05-27T08:48:22.543 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.543 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.543 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.543 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-05-27T08:48:22.543 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-05-27T08:48:22.543 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.544 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.544 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.544 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:48:22.544 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.544 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.544 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.544 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.544 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.545 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-05-27T08:48:22.546 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.546 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.546 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.546 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.546 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.546 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.546 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.547 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.547 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.547 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.547 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.547 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.547 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.547 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.547 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.548 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.549 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.549 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.549 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.549 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.549 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.549 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.549 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.549 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.550 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.550 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.550 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.550 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.550 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.550 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.550 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.551 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.552 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.552 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.552 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.552 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.552 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.552 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.552 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.552 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.553 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.554 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.554 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.554 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.554 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.554 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.554 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.554 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.554 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.555 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.556 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.557 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.558 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-05-27T08:48:22.558 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-05-27T08:48:22.558 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.558 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.558 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.558 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.558 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.558 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-05-27T08:48:22.559 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.560 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.560 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.560 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.560 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.560 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.560 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.560 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.561 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.562 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.562 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.562 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.562 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.562 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:48:22.562 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.562 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.563 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.563 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.563 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-05-27T08:48:22.563 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-05-27T08:48:22.563 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.563 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.563 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.563 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.564 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.564 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.564 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.564 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:48:22.564 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.564 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.564 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.565 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-05-27T08:48:22.565 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-05-27T08:48:22.565 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.565 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.565 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.565 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.565 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.565 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.566 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.566 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:48:22.566 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.566 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.566 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.566 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.566 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-05-27T08:48:22.566 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-05-27T08:48:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.567 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.568 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.569 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.569 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.569 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.569 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.569 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.569 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.569 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.569 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.570 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.570 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-05-27T08:48:22.570 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-05-27T08:48:22.570 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.570 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.570 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.570 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:48:22.570 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.571 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.571 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.571 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.571 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.571 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.571 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.571 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-05-27T08:48:22.572 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-05-27T08:48:22.572 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.572 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.572 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.572 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.572 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.572 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.572 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.573 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.574 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.574 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-05-27T08:48:22.574 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-05-27T08:48:22.574 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.574 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.574 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.574 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.574 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.575 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.575 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.575 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-05-27T08:48:22.575 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-05-27T08:48:22.575 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.575 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.575 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.575 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-05-27T08:48:22.576 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.576 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.576 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.576 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.576 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.576 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.576 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.576 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.577 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-05-27T08:48:22.577 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-05-27T08:48:22.577 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.577 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.577 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.577 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.577 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.577 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.578 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.578 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.578 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.578 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.578 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.578 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.578 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-05-27T08:48:22.578 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-05-27T08:48:22.579 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.579 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.579 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.579 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.579 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.579 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.579 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.579 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.580 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.580 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.580 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.580 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.580 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.580 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.580 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.580 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.581 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.581 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.581 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.581 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.581 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.581 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.581 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.581 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.582 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.582 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.582 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.582 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.582 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.582 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.582 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.582 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.583 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.583 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.583 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.583 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.583 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.583 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.583 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.583 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.584 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.584 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.584 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.584 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.584 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.584 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.584 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.584 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.585 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.586 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.586 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.586 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.586 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.586 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.586 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.586 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.586 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.587 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.587 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.587 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.587 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.587 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.587 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.587 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.587 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.588 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.588 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.588 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.588 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.588 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.588 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.588 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.588 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.589 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.589 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-05-27T08:48:22.589 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-05-27T08:48:22.589 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.589 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.589 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.589 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.589 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.590 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.590 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.590 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.590 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.590 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.590 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.590 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.590 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-05-27T08:48:22.591 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-05-27T08:48:22.591 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.591 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.591 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.591 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.591 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.591 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.591 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.592 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.592 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.592 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.592 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.592 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.592 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.592 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.592 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.593 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.593 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.593 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.593 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.593 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.593 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.593 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.593 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.594 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.594 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.594 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.594 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-05-27T08:48:22.594 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-05-27T08:48:22.594 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.594 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.595 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.595 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:48:22.595 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.595 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.595 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.595 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.595 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.595 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-05-27T08:48:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.596 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.597 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.597 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.597 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.597 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.597 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.597 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.597 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.598 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.598 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.598 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.598 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.598 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.598 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.598 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.598 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.599 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.599 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.599 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.599 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.599 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.599 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.599 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.599 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.600 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.600 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.600 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.600 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.600 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.600 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.600 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.600 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.601 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.602 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.602 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.602 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.602 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.602 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.602 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.602 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.603 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.603 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.603 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.603 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.603 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.603 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.603 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.603 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.604 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.605 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.605 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.605 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.605 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.605 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.605 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.605 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.605 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.606 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.607 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.607 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.607 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.607 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.607 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.607 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.607 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.607 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-05-27T08:48:22.608 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.608 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.608 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.608 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.608 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.608 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.608 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.608 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.609 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.610 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.610 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.610 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.610 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.610 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.610 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.610 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.611 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.612 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.612 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.612 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.612 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.612 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.612 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.612 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.612 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.613 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.613 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.613 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.613 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.613 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.613 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.613 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.613 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.614 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.614 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.614 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.614 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.614 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.614 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.614 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.614 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.615 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.615 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.615 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.615 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.615 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.615 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.615 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.615 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.616 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-05-27T08:48:22.617 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-05-27T08:48:22.617 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.617 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.617 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.617 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-05-27T08:48:22.617 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.617 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.617 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.618 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:48:22.618 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.618 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.618 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.618 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.618 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-05-27T08:48:22.618 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-05-27T08:48:22.618 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.619 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-05-27T08:48:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:48:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.620 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.621 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.621 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.621 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.621 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:48:22.621 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.621 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.621 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.621 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.622 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:48:22.622 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:48:22.622 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.622 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.622 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.622 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.622 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.622 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.623 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.623 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:48:22.623 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.623 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.623 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.623 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.623 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.623 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.624 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.624 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.624 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.624 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.624 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.624 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.624 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.624 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-05-27T08:48:22.625 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.626 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.626 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.626 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-05-27T08:48:22.626 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-05-27T08:48:22.626 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:22.626 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.626 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.626 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-05-27T08:48:22.627 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.627 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.627 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.627 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.627 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.627 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.627 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.627 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.628 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.629 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.629 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.629 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.629 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.629 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.629 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:22.629 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:22.629 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.630 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.630 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.630 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:22.630 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.630 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.630 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.630 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.630 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.631 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.631 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.631 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.631 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.631 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.631 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.631 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.631 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.632 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.633 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.633 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.633 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.633 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.633 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.633 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.633 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.633 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.634 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.634 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.634 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.634 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.634 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.634 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.634 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.634 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.635 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.636 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.637 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.637 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.637 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.637 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.637 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.637 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.637 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.638 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.638 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.638 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.638 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.638 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.638 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.638 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.638 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.639 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.639 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.639 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.639 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.639 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.639 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.640 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.641 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.642 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.642 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.642 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.642 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.642 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.642 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.642 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-05-27T08:48:22.643 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.644 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.644 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.644 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.644 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.644 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.644 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.644 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.644 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.645 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.646 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.646 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.646 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.646 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.646 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.646 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.646 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:22.646 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.647 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.647 INFO:teuthology.orchestra.run.smithi077.stdout: "modules": [ 2024-05-27T08:48:22.647 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-05-27T08:48:22.647 INFO:teuthology.orchestra.run.smithi077.stdout: "dashboard", 2024-05-27T08:48:22.647 INFO:teuthology.orchestra.run.smithi077.stdout: "iostat", 2024-05-27T08:48:22.647 INFO:teuthology.orchestra.run.smithi077.stdout: "nfs", 2024-05-27T08:48:22.647 INFO:teuthology.orchestra.run.smithi077.stdout: "restful", 2024-05-27T08:48:22.647 INFO:teuthology.orchestra.run.smithi077.stdout: "snap_schedule" 2024-05-27T08:48:22.648 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.648 INFO:teuthology.orchestra.run.smithi077.stdout: "available_modules": [ 2024-05-27T08:48:22.648 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.648 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "alerts", 2024-05-27T08:48:22.648 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.648 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.648 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.648 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.649 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How frequently to reexamine health status", 2024-05-27T08:48:22.650 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.650 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.650 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.650 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.650 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.650 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.650 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.651 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.652 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.652 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.652 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.652 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.652 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.652 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.652 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.652 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.653 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.654 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.654 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.654 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.654 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.654 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.654 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.654 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.655 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.655 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.655 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.655 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.655 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.655 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.655 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.655 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.656 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.656 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.656 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.656 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.656 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.656 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.656 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.656 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.657 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.658 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.658 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.658 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.658 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.658 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.658 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.658 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.658 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.659 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_destination": { 2024-05-27T08:48:22.659 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_destination", 2024-05-27T08:48:22.659 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.659 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.659 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.659 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.659 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.659 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email address to send alerts to", 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_from_name": { 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_from_name", 2024-05-27T08:48:22.660 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.661 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.661 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.661 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "Ceph", 2024-05-27T08:48:22.661 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.661 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.661 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.661 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Email From: name", 2024-05-27T08:48:22.661 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.662 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.662 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.662 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_host": { 2024-05-27T08:48:22.662 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_host", 2024-05-27T08:48:22.662 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.662 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.662 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.663 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.663 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.663 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.663 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.663 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP server", 2024-05-27T08:48:22.663 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.663 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.663 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_password": { 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_password", 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.664 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.665 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.665 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Password to authenticate with", 2024-05-27T08:48:22.665 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.665 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.665 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.665 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_port": { 2024-05-27T08:48:22.665 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_port", 2024-05-27T08:48:22.666 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.666 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.666 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.666 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "465", 2024-05-27T08:48:22.666 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.666 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.666 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.666 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP port", 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_sender": { 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_sender", 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.667 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.668 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.668 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.668 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.668 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.668 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SMTP envelope sender", 2024-05-27T08:48:22.668 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.668 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.668 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_ssl": { 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_ssl", 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.669 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-27T08:48:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: "smtp_user": { 2024-05-27T08:48:22.670 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smtp_user", 2024-05-27T08:48:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.671 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "User to authenticate as", 2024-05-27T08:48:22.672 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.672 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.672 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.672 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.672 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.672 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.672 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.672 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.673 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.673 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.673 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.673 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.673 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.673 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.673 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.673 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.674 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.674 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.674 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.674 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.674 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.674 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "balancer", 2024-05-27T08:48:22.674 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.674 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "active": { 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "active", 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.675 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.676 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.676 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically balance PGs across cluster", 2024-05-27T08:48:22.676 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.676 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.676 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.676 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.676 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_time": { 2024-05-27T08:48:22.676 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_time", 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0000", 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "beginning time of day to automatically balance", 2024-05-27T08:48:22.677 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: "begin_weekday": { 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "begin_weekday", 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.678 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.679 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_max_iterations": { 2024-05-27T08:48:22.680 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_max_iterations", 2024-05-27T08:48:22.680 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:22.680 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.680 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.680 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "25", 2024-05-27T08:48:22.680 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:22.680 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "250", 2024-05-27T08:48:22.680 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.681 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-27T08:48:22.681 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.681 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.681 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.681 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.681 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_metrics": { 2024-05-27T08:48:22.681 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_metrics", 2024-05-27T08:48:22.681 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.682 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.682 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.682 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "pgs,objects,bytes", 2024-05-27T08:48:22.682 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.682 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.682 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.682 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-27T08:48:22.682 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: "crush_compat_step": { 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crush_compat_step", 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.683 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.5", 2024-05-27T08:48:22.684 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0.001", 2024-05-27T08:48:22.684 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "0.999", 2024-05-27T08:48:22.684 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.684 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "aggressiveness of optimization", 2024-05-27T08:48:22.684 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-27T08:48:22.684 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.684 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.684 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.685 INFO:teuthology.orchestra.run.smithi077.stdout: "end_time": { 2024-05-27T08:48:22.685 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_time", 2024-05-27T08:48:22.685 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.685 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.685 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.685 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2359", 2024-05-27T08:48:22.685 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.685 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.686 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.686 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ending time of day to automatically balance", 2024-05-27T08:48:22.686 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-27T08:48:22.686 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.686 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.686 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.686 INFO:teuthology.orchestra.run.smithi077.stdout: "end_weekday": { 2024-05-27T08:48:22.686 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "end_weekday", 2024-05-27T08:48:22.687 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:22.687 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.687 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.687 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.687 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "0", 2024-05-27T08:48:22.687 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "6", 2024-05-27T08:48:22.687 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.687 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-27T08:48:22.688 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-27T08:48:22.688 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.688 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.688 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.688 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.688 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.688 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.688 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.689 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.689 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.689 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.689 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.689 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.689 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.689 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.689 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.690 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.690 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.690 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.690 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.690 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.690 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.690 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.690 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.691 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.692 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.692 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.692 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.692 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.692 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.692 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.692 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.692 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.693 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.693 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.693 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.693 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.693 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.693 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.693 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.693 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.694 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.694 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.694 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.694 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.694 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.694 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.694 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.694 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.695 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.695 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.695 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.695 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.695 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.695 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.695 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.695 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.696 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.696 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.696 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.696 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.696 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.696 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.696 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.696 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.697 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.697 INFO:teuthology.orchestra.run.smithi077.stdout: "min_score": { 2024-05-27T08:48:22.697 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_score", 2024-05-27T08:48:22.697 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:22.697 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.697 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.697 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.697 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.698 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.698 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.698 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-27T08:48:22.698 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.698 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.698 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.698 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.698 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-05-27T08:48:22.699 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-05-27T08:48:22.699 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.699 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.699 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.699 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "upmap", 2024-05-27T08:48:22.699 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.699 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.699 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.700 INFO:teuthology.orchestra.run.smithi077.stdout: "crush-compat", 2024-05-27T08:48:22.700 INFO:teuthology.orchestra.run.smithi077.stdout: "none", 2024-05-27T08:48:22.700 INFO:teuthology.orchestra.run.smithi077.stdout: "read", 2024-05-27T08:48:22.700 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap", 2024-05-27T08:48:22.700 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap-read" 2024-05-27T08:48:22.700 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.700 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Balancer mode", 2024-05-27T08:48:22.700 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_ids": { 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_ids", 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.701 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.702 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.702 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.702 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.702 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-27T08:48:22.702 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.702 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.702 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.702 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.703 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:22.703 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:22.703 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.703 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.703 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.703 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:22.703 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.703 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.704 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.704 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-27T08:48:22.704 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.704 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.704 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.704 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.704 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.704 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.705 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.705 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.705 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.705 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.705 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.705 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.705 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.705 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.706 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.706 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.706 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.706 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.706 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_deviation": { 2024-05-27T08:48:22.706 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_deviation", 2024-05-27T08:48:22.706 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.706 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.707 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.707 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:22.707 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:22.707 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.707 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.707 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-27T08:48:22.707 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-27T08:48:22.707 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: "upmap_max_optimizations": { 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "upmap_max_optimizations", 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.708 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.709 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.710 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.710 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm", 2024-05-27T08:48:22.710 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.710 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.710 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.710 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_down_multiplier": { 2024-05-27T08:48:22.710 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_down_multiplier", 2024-05-27T08:48:22.710 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:22.711 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.711 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.711 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-05-27T08:48:22.711 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.711 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.711 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.711 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-27T08:48:22.711 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_refresh_rate": { 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_refresh_rate", 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.712 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "20", 2024-05-27T08:48:22.713 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.713 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.713 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.713 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-27T08:48:22.713 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.713 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.713 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.713 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.714 INFO:teuthology.orchestra.run.smithi077.stdout: "agent_starting_port": { 2024-05-27T08:48:22.714 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "agent_starting_port", 2024-05-27T08:48:22.714 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.714 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.714 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.714 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4721", 2024-05-27T08:48:22.714 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.714 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.715 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.715 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-27T08:48:22.715 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.715 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.715 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.715 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.715 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_ptrace": { 2024-05-27T08:48:22.715 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_ptrace", 2024-05-27T08:48:22.716 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.716 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.716 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.716 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.716 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.716 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.716 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.716 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-27T08:48:22.717 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-27T08:48:22.717 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.717 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.717 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.717 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_interval": { 2024-05-27T08:48:22.717 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_interval", 2024-05-27T08:48:22.717 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.717 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.718 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.718 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:22.718 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.718 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.718 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.718 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-27T08:48:22.718 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.718 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: "autotune_memory_target_ratio": { 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "autotune_memory_target_ratio", 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0.7", 2024-05-27T08:48:22.719 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.720 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.720 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.720 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-27T08:48:22.720 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.720 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.720 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.720 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.720 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm_log_destination": { 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cephadm_log_destination", 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.721 INFO:teuthology.orchestra.run.smithi077.stdout: "file", 2024-05-27T08:48:22.722 INFO:teuthology.orchestra.run.smithi077.stdout: "file,syslog", 2024-05-27T08:48:22.722 INFO:teuthology.orchestra.run.smithi077.stdout: "syslog" 2024-05-27T08:48:22.722 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.722 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-27T08:48:22.722 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.722 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.722 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.722 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: "cgroups_split": { 2024-05-27T08:48:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cgroups_split", 2024-05-27T08:48:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.724 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.724 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-27T08:48:22.724 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.724 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.724 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.724 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.724 INFO:teuthology.orchestra.run.smithi077.stdout: "config_checks_enabled": { 2024-05-27T08:48:22.724 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_checks_enabled", 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-27T08:48:22.725 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.726 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.726 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.726 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.726 INFO:teuthology.orchestra.run.smithi077.stdout: "config_dashboard": { 2024-05-27T08:48:22.726 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_dashboard", 2024-05-27T08:48:22.726 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.726 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.726 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.727 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.727 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.727 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.727 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.727 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-27T08:48:22.727 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.727 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.727 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.728 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.728 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_alertmanager": { 2024-05-27T08:48:22.728 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_alertmanager", 2024-05-27T08:48:22.728 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.728 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.728 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.728 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-27T08:48:22.728 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.729 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.729 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.729 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:22.729 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.729 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.729 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.729 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.729 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_base": { 2024-05-27T08:48:22.730 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_base", 2024-05-27T08:48:22.730 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.730 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.730 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.730 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-27T08:48:22.730 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.730 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.730 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Container image name, without the tag", 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_elasticsearch": { 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_elasticsearch", 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.731 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.732 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.732 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-27T08:48:22.732 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.732 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.732 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.732 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "elasticsearch container image", 2024-05-27T08:48:22.732 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.732 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.733 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.733 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.733 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_grafana": { 2024-05-27T08:48:22.733 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_grafana", 2024-05-27T08:48:22.733 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.733 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.733 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.733 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-27T08:48:22.734 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.734 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.734 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.734 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:22.734 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.734 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.734 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.734 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.735 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_haproxy": { 2024-05-27T08:48:22.735 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_haproxy", 2024-05-27T08:48:22.735 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.735 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.735 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.735 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-27T08:48:22.735 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.735 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "HAproxy container image", 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_agent": { 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_agent", 2024-05-27T08:48:22.736 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.737 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.737 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.737 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-27T08:48:22.737 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.737 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.737 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.737 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger agent container image", 2024-05-27T08:48:22.737 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.738 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.738 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.738 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_collector": { 2024-05-27T08:48:22.738 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_collector", 2024-05-27T08:48:22.738 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.738 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-27T08:48:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger collector container image", 2024-05-27T08:48:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.739 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.740 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.740 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.740 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_jaeger_query": { 2024-05-27T08:48:22.740 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_jaeger_query", 2024-05-27T08:48:22.740 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.740 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.740 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.740 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-27T08:48:22.741 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.741 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.741 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.741 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Jaeger query container image", 2024-05-27T08:48:22.741 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.741 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.741 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.741 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.742 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_keepalived": { 2024-05-27T08:48:22.742 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_keepalived", 2024-05-27T08:48:22.742 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.742 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.742 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.742 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-27T08:48:22.742 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.742 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.743 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.743 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Keepalived container image", 2024-05-27T08:48:22.743 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.743 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.743 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.743 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.743 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_loki": { 2024-05-27T08:48:22.743 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_loki", 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Loki container image", 2024-05-27T08:48:22.744 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.745 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.745 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.745 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.745 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_node_exporter": { 2024-05-27T08:48:22.745 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_node_exporter", 2024-05-27T08:48:22.745 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.745 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.745 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.746 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-27T08:48:22.746 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.746 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.746 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.746 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:22.746 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.746 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_nvmeof": { 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_nvmeof", 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-05-27T08:48:22.747 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.748 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.748 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.748 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Nvme-of container image", 2024-05-27T08:48:22.748 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.748 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.748 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.748 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.748 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_prometheus": { 2024-05-27T08:48:22.749 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_prometheus", 2024-05-27T08:48:22.749 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.749 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.749 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.749 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-27T08:48:22.749 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.749 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.750 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.750 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Prometheus container image", 2024-05-27T08:48:22.750 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.750 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.750 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.750 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.750 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_promtail": { 2024-05-27T08:48:22.750 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_promtail", 2024-05-27T08:48:22.751 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.751 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.751 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.751 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-05-27T08:48:22.751 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.751 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.751 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Promtail container image", 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_samba": { 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_samba", 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.752 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.753 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.753 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-27T08:48:22.753 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.753 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.753 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.753 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Samba/SMB container image", 2024-05-27T08:48:22.753 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.753 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.754 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.754 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.754 INFO:teuthology.orchestra.run.smithi077.stdout: "container_image_snmp_gateway": { 2024-05-27T08:48:22.754 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_image_snmp_gateway", 2024-05-27T08:48:22.754 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.754 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.754 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.754 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-27T08:48:22.755 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.755 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.755 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.755 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "SNMP Gateway container image", 2024-05-27T08:48:22.755 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.755 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.755 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.755 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.756 INFO:teuthology.orchestra.run.smithi077.stdout: "container_init": { 2024-05-27T08:48:22.756 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "container_init", 2024-05-27T08:48:22.756 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.756 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.756 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.756 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.756 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.756 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.757 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.757 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Run podman/docker with `--init`", 2024-05-27T08:48:22.757 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.757 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.757 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.757 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.757 INFO:teuthology.orchestra.run.smithi077.stdout: "daemon_cache_timeout": { 2024-05-27T08:48:22.757 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "daemon_cache_timeout", 2024-05-27T08:48:22.758 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.758 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.758 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.758 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:22.758 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.758 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.758 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.759 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-27T08:48:22.759 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.759 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.759 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.759 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.759 INFO:teuthology.orchestra.run.smithi077.stdout: "default_cephadm_command_timeout": { 2024-05-27T08:48:22.759 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_cephadm_command_timeout", 2024-05-27T08:48:22.759 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "900", 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.760 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: "default_registry": { 2024-05-27T08:48:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "default_registry", 2024-05-27T08:48:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "docker.io", 2024-05-27T08:48:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-27T08:48:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.762 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.763 INFO:teuthology.orchestra.run.smithi077.stdout: "device_cache_timeout": { 2024-05-27T08:48:22.763 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_cache_timeout", 2024-05-27T08:48:22.763 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.763 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.763 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.763 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1800", 2024-05-27T08:48:22.763 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.763 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.764 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.764 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache device inventory", 2024-05-27T08:48:22.764 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.764 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.764 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.764 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.764 INFO:teuthology.orchestra.run.smithi077.stdout: "device_enhanced_scan": { 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_enhanced_scan", 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.765 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-27T08:48:22.766 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.766 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.766 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.766 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.766 INFO:teuthology.orchestra.run.smithi077.stdout: "facts_cache_timeout": { 2024-05-27T08:48:22.766 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "facts_cache_timeout", 2024-05-27T08:48:22.766 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.767 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.767 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.767 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:22.767 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.767 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.767 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.767 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "seconds to cache host facts data", 2024-05-27T08:48:22.767 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.768 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.768 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.768 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.768 INFO:teuthology.orchestra.run.smithi077.stdout: "grafana_dashboards_path": { 2024-05-27T08:48:22.768 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "grafana_dashboards_path", 2024-05-27T08:48:22.768 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.768 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.768 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.769 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-27T08:48:22.769 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.769 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.769 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.769 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-27T08:48:22.769 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.769 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.769 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.770 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.770 INFO:teuthology.orchestra.run.smithi077.stdout: "host_check_interval": { 2024-05-27T08:48:22.770 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "host_check_interval", 2024-05-27T08:48:22.770 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.770 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.770 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.770 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:22.770 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.771 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.771 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.771 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to perform a host check", 2024-05-27T08:48:22.771 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.771 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.771 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.771 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.771 INFO:teuthology.orchestra.run.smithi077.stdout: "hw_monitoring": { 2024-05-27T08:48:22.772 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hw_monitoring", 2024-05-27T08:48:22.772 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.772 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.772 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.772 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.772 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.772 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.772 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.773 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-27T08:48:22.773 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.773 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.773 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.773 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.773 INFO:teuthology.orchestra.run.smithi077.stdout: "inventory_list_all": { 2024-05-27T08:48:22.773 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "inventory_list_all", 2024-05-27T08:48:22.773 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.774 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.774 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.774 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.774 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.774 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.774 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.774 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-27T08:48:22.774 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.775 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.775 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.775 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.775 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.775 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.775 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.775 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.775 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.776 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.776 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.776 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.776 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.776 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.776 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.776 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.776 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.777 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.777 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.777 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.777 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.777 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.777 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.777 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.777 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.778 INFO:teuthology.orchestra.run.smithi077.stdout: "log_refresh_metadata": { 2024-05-27T08:48:22.778 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_refresh_metadata", 2024-05-27T08:48:22.778 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.778 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.778 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.778 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.778 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.778 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.779 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.779 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-27T08:48:22.779 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.779 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.779 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.779 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.779 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.779 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.780 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.780 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.780 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.780 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.780 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.780 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.780 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.780 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-27T08:48:22.781 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.781 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.781 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.781 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.781 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.781 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.781 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.781 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.782 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.782 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.782 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.782 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.782 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.782 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.782 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.783 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.784 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.784 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.784 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.784 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.784 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.784 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.784 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.785 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.785 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.785 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.785 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.785 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.785 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.785 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.785 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.786 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-27T08:48:22.786 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-27T08:48:22.786 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.786 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.786 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.786 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.786 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.786 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.787 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.787 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-27T08:48:22.787 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.787 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.787 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.787 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.787 INFO:teuthology.orchestra.run.smithi077.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-27T08:48:22.787 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "*", 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-27T08:48:22.788 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: "max_count_per_host": { 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_count_per_host", 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.789 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.790 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.790 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.790 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.790 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of daemons per service per host", 2024-05-27T08:48:22.790 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.790 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.790 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.790 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.791 INFO:teuthology.orchestra.run.smithi077.stdout: "max_osd_draining_count": { 2024-05-27T08:48:22.791 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_osd_draining_count", 2024-05-27T08:48:22.791 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.791 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.791 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.791 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.791 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.792 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.792 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.792 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-27T08:48:22.792 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.792 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.792 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.792 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.792 INFO:teuthology.orchestra.run.smithi077.stdout: "migration_current": { 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "migration_current", 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.793 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "internal - do not modify", 2024-05-27T08:48:22.794 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.794 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.794 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.794 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.794 INFO:teuthology.orchestra.run.smithi077.stdout: "mode": { 2024-05-27T08:48:22.794 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mode", 2024-05-27T08:48:22.794 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.794 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.795 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.795 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "root", 2024-05-27T08:48:22.795 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.795 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.795 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.795 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm-package", 2024-05-27T08:48:22.795 INFO:teuthology.orchestra.run.smithi077.stdout: "root" 2024-05-27T08:48:22.795 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.796 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "mode for remote execution of cephadm", 2024-05-27T08:48:22.796 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.796 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.796 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.796 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.796 INFO:teuthology.orchestra.run.smithi077.stdout: "oob_default_addr": { 2024-05-27T08:48:22.796 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "oob_default_addr", 2024-05-27T08:48:22.796 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.797 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.797 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.797 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "169.254.1.1", 2024-05-27T08:48:22.797 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.797 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.797 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.797 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-27T08:48:22.797 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.798 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.798 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.798 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.798 INFO:teuthology.orchestra.run.smithi077.stdout: "prometheus_alerts_path": { 2024-05-27T08:48:22.798 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus_alerts_path", 2024-05-27T08:48:22.798 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.798 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.799 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.799 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-27T08:48:22.799 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.799 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.799 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.799 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-27T08:48:22.799 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.799 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.800 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.800 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.800 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_insecure": { 2024-05-27T08:48:22.800 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_insecure", 2024-05-27T08:48:22.800 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.800 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.800 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.800 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.801 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_password": { 2024-05-27T08:48:22.802 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_password", 2024-05-27T08:48:22.802 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.802 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.802 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.802 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.802 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.802 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.802 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.803 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-27T08:48:22.803 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.803 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.803 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.803 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.803 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_url": { 2024-05-27T08:48:22.803 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_url", 2024-05-27T08:48:22.803 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.804 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.804 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.804 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.804 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.804 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.804 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.804 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-27T08:48:22.804 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.805 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.805 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.805 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.805 INFO:teuthology.orchestra.run.smithi077.stdout: "registry_username": { 2024-05-27T08:48:22.805 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "registry_username", 2024-05-27T08:48:22.805 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.805 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.805 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.806 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.806 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.806 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.806 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.806 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-27T08:48:22.806 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.806 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.806 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.807 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.807 INFO:teuthology.orchestra.run.smithi077.stdout: "secure_monitoring_stack": { 2024-05-27T08:48:22.807 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "secure_monitoring_stack", 2024-05-27T08:48:22.807 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.807 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.807 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.807 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.808 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.808 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.808 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.808 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-27T08:48:22.808 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.808 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.808 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.808 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "service_discovery_port": { 2024-05-27T08:48:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "service_discovery_port", 2024-05-27T08:48:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8765", 2024-05-27T08:48:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.809 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.810 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.810 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "cephadm service discovery port", 2024-05-27T08:48:22.810 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.810 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.810 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.810 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.810 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.810 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.811 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.811 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.811 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.811 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.811 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.811 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.811 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.811 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.812 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.812 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.812 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.812 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.812 INFO:teuthology.orchestra.run.smithi077.stdout: "ssh_config_file": { 2024-05-27T08:48:22.812 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssh_config_file", 2024-05-27T08:48:22.812 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.812 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.813 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.813 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.813 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.813 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.813 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.813 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-27T08:48:22.813 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.813 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.814 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.814 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.814 INFO:teuthology.orchestra.run.smithi077.stdout: "use_agent": { 2024-05-27T08:48:22.814 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_agent", 2024-05-27T08:48:22.814 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.814 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.814 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.815 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.816 INFO:teuthology.orchestra.run.smithi077.stdout: "use_repo_digest": { 2024-05-27T08:48:22.816 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "use_repo_digest", 2024-05-27T08:48:22.816 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.816 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.816 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.816 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.816 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.816 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-27T08:48:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_failed_host_check": { 2024-05-27T08:48:22.817 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_failed_host_check", 2024-05-27T08:48:22.818 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.818 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.818 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.818 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.818 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.818 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.818 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.819 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if the host check fails", 2024-05-27T08:48:22.819 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.819 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.819 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.819 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.819 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_daemons": { 2024-05-27T08:48:22.819 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_daemons", 2024-05-27T08:48:22.819 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.820 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.820 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.820 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.820 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.820 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.820 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.820 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-27T08:48:22.820 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.821 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.821 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.821 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.821 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_on_stray_hosts": { 2024-05-27T08:48:22.821 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_on_stray_hosts", 2024-05-27T08:48:22.821 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.821 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.821 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.822 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.822 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.822 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.822 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.822 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-27T08:48:22.822 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.822 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.822 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crash", 2024-05-27T08:48:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.823 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.824 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.824 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.824 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.824 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.824 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.824 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.824 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.824 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.825 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.825 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.825 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.825 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.825 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.825 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.825 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.825 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.826 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.826 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.826 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.826 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.826 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.826 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.826 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.826 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.827 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.827 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.827 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.827 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.827 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.827 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.827 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.827 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.828 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.828 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.828 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.828 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.828 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.828 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.828 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.828 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.829 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.830 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.830 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.830 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.830 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.830 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.830 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.830 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.831 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.831 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.831 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.831 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.831 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.831 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.831 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.831 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: "retain_interval": { 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retain_interval", 2024-05-27T08:48:22.832 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.833 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.833 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.833 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "31536000", 2024-05-27T08:48:22.833 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.833 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.833 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.833 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-27T08:48:22.833 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.834 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.834 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.834 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.834 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.834 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.834 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.834 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.834 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.835 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.835 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.835 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.835 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.835 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.835 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.835 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.835 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.836 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.836 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_recent_interval": { 2024-05-27T08:48:22.836 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_recent_interval", 2024-05-27T08:48:22.836 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.836 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.836 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.836 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1209600", 2024-05-27T08:48:22.837 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.837 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.837 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.837 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-27T08:48:22.837 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.837 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.837 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.837 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.838 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.838 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.838 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.838 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dashboard", 2024-05-27T08:48:22.838 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.838 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.838 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.838 INFO:teuthology.orchestra.run.smithi077.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-27T08:48:22.839 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-27T08:48:22.839 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.839 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.839 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.839 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.839 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.839 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.839 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.840 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.840 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.840 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.840 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.840 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.840 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_HOST": { 2024-05-27T08:48:22.840 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-27T08:48:22.840 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.841 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.842 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.842 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.842 INFO:teuthology.orchestra.run.smithi077.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-27T08:48:22.842 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-27T08:48:22.842 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.842 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.842 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.843 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.844 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_ENABLED": { 2024-05-27T08:48:22.844 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_ENABLED", 2024-05-27T08:48:22.844 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.844 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.844 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.844 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.844 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.845 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.845 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.845 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.845 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.845 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.845 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.845 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.845 INFO:teuthology.orchestra.run.smithi077.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-27T08:48:22.846 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-27T08:48:22.846 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.846 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.846 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.846 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.846 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.846 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.846 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.847 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.847 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.847 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.847 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.847 INFO:teuthology.orchestra.run.smithi077.stdout: "ENABLE_BROWSABLE_API": { 2024-05-27T08:48:22.847 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-27T08:48:22.847 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.848 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.848 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.848 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.848 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.848 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.848 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.848 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.848 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.849 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.849 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.849 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-27T08:48:22.849 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-27T08:48:22.849 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.849 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.849 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.850 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.850 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.850 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.850 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.850 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.850 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.850 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.850 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-27T08:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-27T08:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.851 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.852 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.852 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.852 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.852 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.852 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.852 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.852 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.852 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-27T08:48:22.853 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-27T08:48:22.853 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.853 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.853 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.853 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.853 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.853 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.853 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.854 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.854 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.854 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.854 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.854 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.854 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-27T08:48:22.854 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-27T08:48:22.854 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.855 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.855 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.855 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.855 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.855 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.855 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.855 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.855 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.856 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.856 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.856 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-27T08:48:22.856 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-27T08:48:22.856 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.856 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.856 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.857 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.857 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.857 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.857 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.857 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.857 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.857 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.857 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.858 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.858 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-27T08:48:22.858 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-27T08:48:22.858 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.858 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.858 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.858 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.859 INFO:teuthology.orchestra.run.smithi077.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.860 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.861 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.861 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.861 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.861 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.861 INFO:teuthology.orchestra.run.smithi077.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-27T08:48:22.861 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-27T08:48:22.861 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.861 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.862 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.862 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.862 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.862 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.862 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.862 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.862 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.862 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.863 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.863 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.863 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_PASSWORD": { 2024-05-27T08:48:22.863 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-27T08:48:22.863 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.863 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.863 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.863 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.864 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-27T08:48:22.865 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-27T08:48:22.865 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.865 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.865 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.865 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.865 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.865 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.865 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.866 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.866 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.866 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.866 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.866 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.866 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_URL": { 2024-05-27T08:48:22.866 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_URL", 2024-05-27T08:48:22.866 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.867 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.867 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.867 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.867 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.867 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.867 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.867 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.867 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_API_USERNAME": { 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.868 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:22.869 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.869 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.869 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.869 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.869 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.869 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.869 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.870 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-27T08:48:22.870 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-27T08:48:22.870 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.870 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.870 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.870 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.870 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.870 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.871 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.871 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.871 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.871 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.871 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.871 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.871 INFO:teuthology.orchestra.run.smithi077.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-27T08:48:22.871 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-27T08:48:22.872 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.872 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.872 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.872 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.872 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.872 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.872 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.872 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.873 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.873 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.873 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.873 INFO:teuthology.orchestra.run.smithi077.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-27T08:48:22.873 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-27T08:48:22.873 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.874 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.874 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.874 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.874 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.874 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.874 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.874 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.874 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.875 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.875 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.875 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.875 INFO:teuthology.orchestra.run.smithi077.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-27T08:48:22.875 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-27T08:48:22.875 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.875 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.875 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.876 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.876 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.876 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.876 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.876 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.876 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.876 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.876 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "[]", 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.877 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.878 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.878 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.878 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.878 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.878 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.878 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.878 INFO:teuthology.orchestra.run.smithi077.stdout: "MULTICLUSTER_CONFIG": { 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "{}", 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.879 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.880 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.880 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.880 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.880 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.880 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_HOST": { 2024-05-27T08:48:22.880 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-27T08:48:22.880 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.880 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.881 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.881 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.881 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.881 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.881 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.881 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.881 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.881 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.882 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.882 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.882 INFO:teuthology.orchestra.run.smithi077.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-27T08:48:22.882 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-27T08:48:22.882 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.882 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.882 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.883 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.884 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-27T08:48:22.884 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-27T08:48:22.884 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.884 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.884 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.884 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.884 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.884 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.885 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.885 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.885 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.885 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.885 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.885 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.885 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-27T08:48:22.885 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-27T08:48:22.886 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.886 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.886 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.886 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.886 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.886 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.886 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.886 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.887 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.888 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.888 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.888 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.888 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.888 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.888 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.888 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.888 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.889 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.889 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-27T08:48:22.889 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-27T08:48:22.889 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.889 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.889 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.889 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.889 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.890 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.890 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.890 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.890 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.890 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.890 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.890 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.890 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-27T08:48:22.891 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-27T08:48:22.891 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.891 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.891 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.891 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.891 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.891 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.891 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.892 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.892 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.892 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.892 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.892 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.892 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-27T08:48:22.892 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-27T08:48:22.892 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.893 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.893 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.893 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.893 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.893 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.893 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.893 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.893 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.894 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.894 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.894 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.894 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-27T08:48:22.894 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-27T08:48:22.894 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.894 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.894 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.895 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.895 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.895 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.895 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.895 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.895 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.895 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_ENABLED": { 2024-05-27T08:48:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-27T08:48:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.896 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.897 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.897 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.897 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.897 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.897 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.897 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.897 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.897 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.898 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-27T08:48:22.898 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-27T08:48:22.898 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.898 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.898 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.898 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-27T08:48:22.898 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.898 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-27T08:48:22.899 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-27T08:48:22.900 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.900 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.900 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.900 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.900 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.900 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.900 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.900 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.901 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.901 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.901 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.901 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.901 INFO:teuthology.orchestra.run.smithi077.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-27T08:48:22.901 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-27T08:48:22.901 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.902 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.902 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.902 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8", 2024-05-27T08:48:22.902 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.902 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.902 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.902 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.902 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.903 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.903 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.903 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.903 INFO:teuthology.orchestra.run.smithi077.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-27T08:48:22.903 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-27T08:48:22.903 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.903 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.903 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.904 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "45", 2024-05-27T08:48:22.904 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.904 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.904 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.904 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.904 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.904 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.904 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ACCESS_KEY": { 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.905 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.906 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.906 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.906 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.906 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.906 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.906 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.906 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "admin", 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.907 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.908 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.908 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.908 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.908 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.908 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SECRET_KEY": { 2024-05-27T08:48:22.908 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-27T08:48:22.908 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.908 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.909 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.909 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.909 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.909 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.909 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.909 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.909 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.909 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.910 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.910 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.910 INFO:teuthology.orchestra.run.smithi077.stdout: "RGW_API_SSL_VERIFY": { 2024-05-27T08:48:22.910 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-27T08:48:22.910 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.910 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.910 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.910 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.911 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.911 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.911 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.911 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.911 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.911 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.911 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.911 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "UNSAFE_TLS_v1_2": { 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.912 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.913 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.913 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.913 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.913 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.913 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.913 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-27T08:48:22.913 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-27T08:48:22.913 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.914 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.914 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.914 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.914 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.914 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.914 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.914 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.915 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.916 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.917 INFO:teuthology.orchestra.run.smithi077.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-27T08:48:22.917 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-27T08:48:22.917 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.917 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.917 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.917 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:22.917 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.917 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.918 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.918 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.918 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.918 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.918 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.918 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.918 INFO:teuthology.orchestra.run.smithi077.stdout: "cross_origin_url": { 2024-05-27T08:48:22.918 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cross_origin_url", 2024-05-27T08:48:22.919 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.919 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.919 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.919 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.919 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.919 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.919 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.919 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: "crt_file": { 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "crt_file", 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.920 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.921 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.921 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.921 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.921 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.921 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.921 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.921 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.921 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.922 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.922 INFO:teuthology.orchestra.run.smithi077.stdout: "debug": { 2024-05-27T08:48:22.922 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "debug", 2024-05-27T08:48:22.922 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.922 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.922 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.922 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.922 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Enable/disable debug options", 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: "jwt_token_ttl": { 2024-05-27T08:48:22.923 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "jwt_token_ttl", 2024-05-27T08:48:22.924 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.924 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.924 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.924 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "28800", 2024-05-27T08:48:22.924 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.924 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.924 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.924 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.925 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.926 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.926 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.926 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.926 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.926 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.926 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.926 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.926 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.927 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.928 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.928 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.928 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.928 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.928 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.928 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.928 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.928 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.929 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.929 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.929 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.929 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.929 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.929 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.929 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.929 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.930 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.930 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.930 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.930 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.930 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.930 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.930 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.931 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.932 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.932 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.932 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.932 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.932 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.932 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.932 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.932 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.933 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.933 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.933 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.933 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.933 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.933 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.933 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.933 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.934 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.934 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.934 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.934 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.934 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.934 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.934 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.934 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: "motd": { 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "motd", 2024-05-27T08:48:22.935 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "The message of the day", 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.936 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect_resolve_ip_addr": { 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "redirect_resolve_ip_addr", 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.937 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.938 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.938 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.938 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.938 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.938 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.938 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.938 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.938 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:22.939 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:22.939 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.939 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.939 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.939 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-05-27T08:48:22.939 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.939 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.939 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.940 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.940 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.940 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.940 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.940 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.940 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:22.940 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:22.940 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8080", 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.941 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.942 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.942 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.942 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.942 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.942 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.942 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.942 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.942 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.943 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.943 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.943 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.943 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.943 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.943 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.943 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.943 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.944 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-05-27T08:48:22.944 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-05-27T08:48:22.944 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.944 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.944 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.944 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.944 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.944 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.945 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.945 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.945 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.945 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.945 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.945 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.945 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl_server_port": { 2024-05-27T08:48:22.945 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl_server_port", 2024-05-27T08:48:22.946 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.946 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.946 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.946 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8443", 2024-05-27T08:48:22.946 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.946 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.946 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.946 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-05-27T08:48:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-05-27T08:48:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.947 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "redirect", 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: "redirect" 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.948 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.949 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.949 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.949 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.949 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.949 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-05-27T08:48:22.949 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-05-27T08:48:22.949 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.949 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.950 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.950 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-05-27T08:48:22.950 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-05-27T08:48:22.950 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-05-27T08:48:22.950 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.950 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.950 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.950 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.951 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.951 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.951 INFO:teuthology.orchestra.run.smithi077.stdout: "url_prefix": { 2024-05-27T08:48:22.951 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url_prefix", 2024-05-27T08:48:22.951 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.951 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.951 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.951 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.952 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.952 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.952 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.952 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.952 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.952 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.952 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.952 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_monitoring": { 2024-05-27T08:48:22.953 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_monitoring", 2024-05-27T08:48:22.954 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.954 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.954 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.954 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.954 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.954 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.954 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.954 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "monitor device health metrics", 2024-05-27T08:48:22.955 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.955 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.955 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.955 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.955 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.955 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.955 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.956 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.957 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.957 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.957 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.957 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.957 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.957 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.957 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.958 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.958 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.958 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.958 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.958 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.958 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.958 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.958 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.959 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.959 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.959 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.959 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.959 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.959 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.959 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.959 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.960 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.961 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.962 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.962 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.962 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.962 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.962 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.962 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.962 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.962 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.963 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.963 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.963 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.963 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.963 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.963 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.963 INFO:teuthology.orchestra.run.smithi077.stdout: "mark_out_threshold": { 2024-05-27T08:48:22.964 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mark_out_threshold", 2024-05-27T08:48:22.964 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.964 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.964 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.964 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "2419200", 2024-05-27T08:48:22.964 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.964 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.964 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: "pool_name": { 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pool_name", 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.965 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.966 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.966 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "device_health_metrics", 2024-05-27T08:48:22.966 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.966 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.966 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.966 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-27T08:48:22.966 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.966 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: "retention_period": { 2024-05-27T08:48:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "retention_period", 2024-05-27T08:48:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.967 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15552000", 2024-05-27T08:48:22.968 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.968 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.968 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.968 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long to retain device health metrics", 2024-05-27T08:48:22.968 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.968 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.968 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_frequency": { 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_frequency", 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.969 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.970 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-27T08:48:22.970 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.970 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.970 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.970 INFO:teuthology.orchestra.run.smithi077.stdout: "self_heal": { 2024-05-27T08:48:22.970 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "self_heal", 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-27T08:48:22.971 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.972 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.972 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:22.972 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:22.972 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.972 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.972 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.973 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:22.973 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.973 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.973 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.973 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how frequently to wake up and check device health", 2024-05-27T08:48:22.973 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.973 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.973 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.974 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.974 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.974 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.974 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.974 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.974 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.974 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.975 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.975 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.975 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.975 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.975 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.975 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.975 INFO:teuthology.orchestra.run.smithi077.stdout: "warn_threshold": { 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "warn_threshold", 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7257600", 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.976 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-27T08:48:22.977 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.977 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.977 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.977 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.977 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.977 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.977 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "diskprediction_local", 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.978 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.979 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.980 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.981 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.982 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.982 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.982 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.982 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.982 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:22.982 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:22.982 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.982 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.983 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.983 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:22.983 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.983 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.983 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:22.983 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:22.983 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:22.983 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:22.984 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:22.984 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:22.984 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:22.984 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:22.984 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.984 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.984 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.984 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.985 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.985 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:22.985 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:22.985 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:22.985 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.985 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.985 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:22.985 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.986 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.986 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.986 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.986 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.986 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.986 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.986 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.986 INFO:teuthology.orchestra.run.smithi077.stdout: "predict_interval": { 2024-05-27T08:48:22.987 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predict_interval", 2024-05-27T08:48:22.987 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.987 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.987 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.987 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "86400", 2024-05-27T08:48:22.987 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.987 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.987 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.988 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.988 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.988 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.988 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.988 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.988 INFO:teuthology.orchestra.run.smithi077.stdout: "predictor_model": { 2024-05-27T08:48:22.988 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "predictor_model", 2024-05-27T08:48:22.988 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "prophetstor", 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.989 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.990 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.990 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.990 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:22.990 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:22.990 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.990 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.990 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.990 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "600", 2024-05-27T08:48:22.991 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.991 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.991 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.991 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.991 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.991 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.991 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.992 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:22.992 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:22.992 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.992 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:22.992 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:22.992 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:22.992 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.992 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.993 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:22.994 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "influx", 2024-05-27T08:48:22.994 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": false, 2024-05-27T08:48:22.994 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "influxdb python module not found", 2024-05-27T08:48:22.994 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:22.994 INFO:teuthology.orchestra.run.smithi077.stdout: "batch_size": { 2024-05-27T08:48:22.994 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "batch_size", 2024-05-27T08:48:22.994 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:22.994 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.995 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.995 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5000", 2024-05-27T08:48:22.995 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.995 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.995 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.995 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-27T08:48:22.995 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.995 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.996 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.996 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.996 INFO:teuthology.orchestra.run.smithi077.stdout: "database": { 2024-05-27T08:48:22.996 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "database", 2024-05-27T08:48:22.996 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.996 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.996 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.996 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "ceph", 2024-05-27T08:48:22.997 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.997 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.997 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.997 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-27T08:48:22.997 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.997 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.997 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.997 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.998 INFO:teuthology.orchestra.run.smithi077.stdout: "hostname": { 2024-05-27T08:48:22.998 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "hostname", 2024-05-27T08:48:22.998 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:22.998 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:22.998 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:22.998 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:22.998 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:22.998 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:22.999 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:22.999 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server hostname", 2024-05-27T08:48:22.999 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:22.999 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:22.999 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:22.999 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:22.999 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:22.999 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:23.000 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:23.000 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.000 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.000 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "30", 2024-05-27T08:48:23.000 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "5", 2024-05-27T08:48:23.000 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.000 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.000 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-27T08:48:23.001 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.001 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.001 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.001 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.001 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.001 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.001 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.001 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.002 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.002 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.002 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.002 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.002 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.002 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.002 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.002 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.003 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.004 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.005 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.005 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.005 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.005 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.005 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.005 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.005 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.005 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.006 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.006 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.006 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.006 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.006 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.006 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.006 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.006 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.007 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.008 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.008 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.008 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.008 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.008 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.008 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.008 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.008 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: "password": { 2024-05-27T08:48:23.009 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "password", 2024-05-27T08:48:23.010 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.010 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.010 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.010 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.010 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.010 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.010 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "password of InfluxDB server user", 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: "port": { 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "port", 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.011 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "8086", 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "InfluxDB server port", 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.012 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.013 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.014 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.014 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.014 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.014 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.014 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.014 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.014 INFO:teuthology.orchestra.run.smithi077.stdout: "ssl": { 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ssl", 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "false", 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.015 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:48:23.016 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.016 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.016 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.016 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.016 INFO:teuthology.orchestra.run.smithi077.stdout: "threads": { 2024-05-27T08:48:23.016 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threads", 2024-05-27T08:48:23.016 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.016 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "32", 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.017 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.018 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.018 INFO:teuthology.orchestra.run.smithi077.stdout: "username": { 2024-05-27T08:48:23.018 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "username", 2024-05-27T08:48:23.018 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.018 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.018 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.018 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.018 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "username of InfluxDB server user", 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: "verify_ssl": { 2024-05-27T08:48:23.019 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "verify_ssl", 2024-05-27T08:48:23.020 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.020 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.020 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.020 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "true", 2024-05-27T08:48:23.020 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.020 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.020 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.020 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "insights", 2024-05-27T08:48:23.021 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.022 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.022 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.022 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.022 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.022 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.022 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.022 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.023 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.023 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.023 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.023 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.023 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.023 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.023 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.023 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.024 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.024 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.024 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.024 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.024 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.024 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.024 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.024 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.025 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.026 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.026 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.026 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.026 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.026 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.026 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.026 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.026 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.027 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.027 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.027 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.027 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.027 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.027 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.027 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.027 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.028 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.029 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.029 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.029 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.029 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.029 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.029 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.029 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.029 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.030 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.030 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.030 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.030 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.030 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.030 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.030 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.030 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.031 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.032 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "iostat", 2024-05-27T08:48:23.033 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.033 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.033 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.033 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.033 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.033 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.033 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.033 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.034 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.034 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.034 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.034 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.034 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.034 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.034 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.034 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.035 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.035 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.035 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.035 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.035 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.035 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.035 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.035 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.036 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.037 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.038 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.038 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.038 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.038 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.038 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.038 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.038 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.038 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.039 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.039 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.039 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.039 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.039 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.039 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.039 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.039 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.040 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.041 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.041 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.041 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.041 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.041 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.041 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.041 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.041 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.042 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.042 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.042 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.042 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.042 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.042 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.042 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.042 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.043 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.043 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.043 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.043 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.043 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.043 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.043 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.043 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "k8sevents", 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_event_retention_days": { 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "ceph_event_retention_days", 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.044 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "7", 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.045 INFO:teuthology.orchestra.run.smithi077.stdout: "config_check_secs": { 2024-05-27T08:48:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "config_check_secs", 2024-05-27T08:48:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "10", 2024-05-27T08:48:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.046 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.047 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-27T08:48:23.047 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.047 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.047 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.047 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.047 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.047 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.047 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.048 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.048 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.048 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.048 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.048 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.048 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.048 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.048 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.049 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.050 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.051 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.052 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.053 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.053 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.053 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.053 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.053 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.053 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.053 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.053 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.054 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.054 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.054 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.054 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.054 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.054 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.054 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.054 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.055 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.055 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.055 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.055 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.055 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.055 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.055 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.055 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.056 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.057 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.058 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "localpool", 2024-05-27T08:48:23.058 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.058 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.058 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.058 INFO:teuthology.orchestra.run.smithi077.stdout: "failure_domain": { 2024-05-27T08:48:23.058 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "failure_domain", 2024-05-27T08:48:23.058 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "host", 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "failure domain for any created local pool", 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-27T08:48:23.059 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.060 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.061 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.062 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.062 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.062 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.062 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.062 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.062 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.062 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.062 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.063 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.064 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.064 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.064 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.064 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.064 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.064 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.064 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.064 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.065 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.066 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.066 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.066 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.066 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.066 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.066 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.066 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.066 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.067 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.067 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.067 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.067 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.067 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.067 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.067 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.067 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: "min_size": { 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "min_size", 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.068 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default min_size for any created local pool", 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.069 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.070 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.070 INFO:teuthology.orchestra.run.smithi077.stdout: "num_rep": { 2024-05-27T08:48:23.070 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "num_rep", 2024-05-27T08:48:23.070 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.070 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.070 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.070 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3", 2024-05-27T08:48:23.070 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.071 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.071 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.071 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default replica count for any created local pool", 2024-05-27T08:48:23.071 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.071 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.071 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.071 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.071 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_num": { 2024-05-27T08:48:23.072 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_num", 2024-05-27T08:48:23.072 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.072 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.072 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.072 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "128", 2024-05-27T08:48:23.072 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.072 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.072 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.073 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "default pg_num for any created local pool", 2024-05-27T08:48:23.073 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.073 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.073 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.073 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.073 INFO:teuthology.orchestra.run.smithi077.stdout: "prefix": { 2024-05-27T08:48:23.073 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prefix", 2024-05-27T08:48:23.073 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.074 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.074 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.074 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.074 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.074 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.074 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.074 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "name prefix for any created local pool", 2024-05-27T08:48:23.074 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.075 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.075 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.075 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.075 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.075 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.075 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.075 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.075 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.076 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.077 INFO:teuthology.orchestra.run.smithi077.stdout: "subtree": { 2024-05-27T08:48:23.077 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "subtree", 2024-05-27T08:48:23.077 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.077 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.077 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.077 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "rack", 2024-05-27T08:48:23.077 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.077 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.078 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.078 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-27T08:48:23.078 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-27T08:48:23.078 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.078 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.078 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.078 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.078 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mds_autoscaler", 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.079 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.080 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.081 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.081 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.081 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.081 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.081 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.081 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.081 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.081 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.082 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.082 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.082 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.082 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.082 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.082 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.082 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.082 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.083 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.083 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.083 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.083 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.083 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.083 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.083 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.083 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.084 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.085 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.086 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.086 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.086 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.086 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.086 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.086 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.086 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.086 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.087 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.087 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.087 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.087 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.087 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.087 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.087 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.087 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.088 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.088 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.088 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.088 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.088 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.088 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.088 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.088 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.089 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.089 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.089 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.089 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.089 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.089 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.089 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.090 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirroring", 2024-05-27T08:48:23.090 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.090 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.090 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.090 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.090 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.090 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.090 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.091 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.091 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.091 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.091 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.091 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.091 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.091 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.091 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.092 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.092 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.092 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.092 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.092 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.092 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.092 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.092 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.093 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.094 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.094 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.094 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.094 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.094 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.094 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.094 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.094 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.095 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.095 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.095 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.095 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.095 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.095 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.095 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.095 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.096 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.096 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.096 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.096 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.096 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.096 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.096 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.096 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.097 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.097 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.097 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.097 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.097 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.097 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.097 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.097 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.098 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.098 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.098 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.098 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.098 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.098 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.098 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.099 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.099 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.099 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.099 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.099 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.099 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.099 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.099 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.100 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.100 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.100 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.100 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.100 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.100 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.100 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.100 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "nfs", 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.101 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.102 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.102 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.102 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.102 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.102 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.102 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.102 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.103 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.104 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.105 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.105 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.105 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.105 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.105 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.105 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.105 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.105 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.106 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.107 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.107 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.107 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.107 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.107 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.107 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.107 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.107 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.108 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.109 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.109 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.109 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.109 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.109 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.109 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.109 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.109 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.110 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.110 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.110 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.110 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.110 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.110 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.110 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.110 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.111 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: "fail_fs": { 2024-05-27T08:48:23.112 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "fail_fs", 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-27T08:48:23.113 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.114 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.114 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.114 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.114 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.114 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.114 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.114 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.114 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.115 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.115 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.115 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.115 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.115 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.115 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.115 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.115 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.116 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.117 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.117 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.117 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.117 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.117 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.117 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.117 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.117 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.118 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.118 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.118 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.118 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.118 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.118 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.118 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.118 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.119 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.119 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.119 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.119 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.119 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.119 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.119 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.119 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.120 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.120 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.120 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.120 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.120 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.120 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.120 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.120 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.121 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.122 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.122 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.122 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.122 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.122 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.122 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.122 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.122 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator": { 2024-05-27T08:48:23.123 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "orchestrator", 2024-05-27T08:48:23.123 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.123 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.123 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.123 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.123 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.123 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.123 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.124 INFO:teuthology.orchestra.run.smithi077.stdout: "cephadm", 2024-05-27T08:48:23.124 INFO:teuthology.orchestra.run.smithi077.stdout: "rook", 2024-05-27T08:48:23.124 INFO:teuthology.orchestra.run.smithi077.stdout: "test_orchestrator" 2024-05-27T08:48:23.124 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.124 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Orchestrator backend", 2024-05-27T08:48:23.124 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.124 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.124 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.125 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.125 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.125 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.125 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.125 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.125 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.125 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.125 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.126 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.126 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.126 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.126 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.126 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.126 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.126 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.126 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.127 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.127 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.127 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_perf_query", 2024-05-27T08:48:23.127 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.127 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.127 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.127 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.127 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.128 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.128 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.128 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.128 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.128 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.128 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.128 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.128 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.129 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.129 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.129 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.129 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.129 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.129 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.129 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.129 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.130 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.130 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.130 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.130 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.130 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.130 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.130 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.130 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.131 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.131 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.131 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.131 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.131 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.131 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.131 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.131 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.132 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.132 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.132 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.132 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.132 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.132 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.132 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.132 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.133 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.133 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.133 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.133 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.133 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.133 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.133 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.133 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.134 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.134 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.134 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.134 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.134 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.134 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.134 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.134 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.135 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.135 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.135 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.135 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.135 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.135 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.135 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.135 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.136 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.136 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.136 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.136 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.136 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.136 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.136 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.137 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.138 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.138 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.138 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.138 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "osd_support", 2024-05-27T08:48:23.138 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.138 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.139 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.140 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.140 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.140 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.140 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.140 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.140 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.140 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.140 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.141 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.142 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.142 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.142 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.142 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.142 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.142 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.142 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.142 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.143 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.143 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.143 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.143 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.143 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.143 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.143 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.144 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.145 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.146 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.146 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.146 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.146 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.146 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.146 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.146 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.147 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.148 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.148 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.148 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.148 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.148 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.148 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.148 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.148 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.149 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.149 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.149 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.149 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.149 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.149 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "pg_autoscaler", 2024-05-27T08:48:23.149 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.149 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.150 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.150 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.150 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.150 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.150 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.150 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.150 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.150 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.151 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.152 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.153 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.154 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.154 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.154 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.154 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.154 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.154 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.154 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.154 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.155 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.156 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.156 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.156 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.156 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.156 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.156 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.156 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.156 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.157 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.158 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.158 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.158 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:23.158 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:23.158 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:23.158 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.158 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.158 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:23.159 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.159 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.159 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.159 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.159 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.159 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.159 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.160 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.160 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.160 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.160 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.160 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.160 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.160 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.160 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: "threshold": { 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "threshold", 2024-05-27T08:48:23.161 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "3.0", 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1.0", 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "scaling threshold", 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-27T08:48:23.162 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "progress", 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.163 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_pg_recovery_event": { 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_pg_recovery_event", 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.164 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.165 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-27T08:48:23.165 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.165 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.165 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.165 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-05-27T08:48:23.165 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-05-27T08:48:23.165 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.166 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.166 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.166 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.166 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.166 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.166 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.166 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.166 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.167 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.168 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.168 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.168 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.168 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.168 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.168 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.168 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.168 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.169 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.170 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.170 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.170 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.170 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.170 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.170 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.170 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.170 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.171 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.172 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.172 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.172 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.172 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.172 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.172 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.172 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.172 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.173 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.173 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.173 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.173 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.173 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.173 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.173 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.174 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.175 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.175 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.175 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.175 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.175 INFO:teuthology.orchestra.run.smithi077.stdout: "max_completed_events": { 2024-05-27T08:48:23.175 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_completed_events", 2024-05-27T08:48:23.175 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.176 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.176 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.176 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "50", 2024-05-27T08:48:23.176 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.176 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.176 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.176 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "number of past completed events to remember", 2024-05-27T08:48:23.176 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: "sleep_interval": { 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sleep_interval", 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.177 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "5", 2024-05-27T08:48:23.178 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.178 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.178 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.178 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "how long the module is going to sleep", 2024-05-27T08:48:23.178 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.178 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.178 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.179 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.179 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.179 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.179 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.179 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.182 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.182 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.182 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.183 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.183 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.183 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.183 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.183 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.183 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.184 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.184 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.184 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "prometheus", 2024-05-27T08:48:23.184 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.184 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.184 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.184 INFO:teuthology.orchestra.run.smithi077.stdout: "cache": { 2024-05-27T08:48:23.185 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "cache", 2024-05-27T08:48:23.185 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.185 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.185 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.185 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.185 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.185 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.185 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.186 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.186 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.186 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.186 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.186 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.186 INFO:teuthology.orchestra.run.smithi077.stdout: "exclude_perf_counters": { 2024-05-27T08:48:23.186 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "exclude_perf_counters", 2024-05-27T08:48:23.187 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.187 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.187 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.187 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.187 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.187 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.187 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.188 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-27T08:48:23.188 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-27T08:48:23.188 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.188 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.188 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.188 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.188 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.189 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.189 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.189 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.189 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.189 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.189 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.189 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.190 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.190 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.190 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.190 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.190 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.190 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.190 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.191 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.191 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.191 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.192 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.192 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.192 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.192 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.192 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.192 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.192 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.193 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.193 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.193 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.193 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.193 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.193 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.193 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.193 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.194 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.194 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.194 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.194 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.194 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.194 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.195 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.195 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.195 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.195 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.195 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.195 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.195 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.195 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.196 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.196 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.196 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.196 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.196 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.196 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.196 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.197 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.197 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.197 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.197 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.197 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.197 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.197 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.198 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.198 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.198 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.198 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.198 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.198 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.198 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.198 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.199 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools": { 2024-05-27T08:48:23.199 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools", 2024-05-27T08:48:23.199 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.199 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.199 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.199 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.199 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.199 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.200 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.200 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.200 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.200 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.200 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.200 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.200 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-27T08:48:23.201 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-27T08:48:23.201 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.201 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.201 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.201 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "300", 2024-05-27T08:48:23.201 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.201 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.202 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.202 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.202 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.202 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.202 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.202 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.202 INFO:teuthology.orchestra.run.smithi077.stdout: "scrape_interval": { 2024-05-27T08:48:23.203 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "scrape_interval", 2024-05-27T08:48:23.203 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:23.203 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.203 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.203 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15.0", 2024-05-27T08:48:23.203 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.203 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.203 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.204 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.204 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.204 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.204 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.204 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.204 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:23.204 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:23.205 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.205 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.205 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.205 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "::", 2024-05-27T08:48:23.205 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.205 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.205 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.206 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-27T08:48:23.206 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.206 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.206 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.206 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.206 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:23.206 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:23.207 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.207 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.207 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.207 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "9283", 2024-05-27T08:48:23.207 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.207 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.207 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.208 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-27T08:48:23.208 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.208 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.208 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.208 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.208 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.208 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.208 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.209 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.209 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.209 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.209 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.209 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.209 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.209 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.210 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.210 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.210 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.210 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.210 INFO:teuthology.orchestra.run.smithi077.stdout: "stale_cache_strategy": { 2024-05-27T08:48:23.210 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stale_cache_strategy", 2024-05-27T08:48:23.210 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.211 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.211 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.211 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "log", 2024-05-27T08:48:23.211 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.211 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.211 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.211 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.211 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.212 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.212 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.212 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.212 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_behaviour": { 2024-05-27T08:48:23.212 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_behaviour", 2024-05-27T08:48:23.212 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.212 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.213 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.213 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "default", 2024-05-27T08:48:23.213 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.213 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.213 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.213 INFO:teuthology.orchestra.run.smithi077.stdout: "default", 2024-05-27T08:48:23.213 INFO:teuthology.orchestra.run.smithi077.stdout: "error" 2024-05-27T08:48:23.214 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.214 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.214 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.214 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.214 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.214 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.214 INFO:teuthology.orchestra.run.smithi077.stdout: "standby_error_status_code": { 2024-05-27T08:48:23.215 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "standby_error_status_code", 2024-05-27T08:48:23.215 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.215 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.215 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.215 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "500", 2024-05-27T08:48:23.215 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "400", 2024-05-27T08:48:23.215 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "599", 2024-05-27T08:48:23.216 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.216 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.216 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.216 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.216 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.216 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.216 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.216 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.217 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.217 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-05-27T08:48:23.217 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.217 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.217 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.217 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.217 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.218 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.218 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.218 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.218 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.218 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.218 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.218 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.219 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.220 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.220 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.220 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.220 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.220 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.220 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.220 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.221 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.221 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.221 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.221 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.221 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.221 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.221 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.222 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.222 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.222 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.222 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.222 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.222 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.222 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.223 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.223 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.223 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.223 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.223 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.223 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.223 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.223 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.224 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.224 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.224 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.224 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.224 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.224 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.224 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.225 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.225 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.225 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.225 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.225 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.225 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.225 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.226 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.226 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.226 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.226 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.226 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.226 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.226 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.227 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.227 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.227 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.227 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_snap_create": { 2024-05-27T08:48:23.227 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_snap_create", 2024-05-27T08:48:23.227 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.227 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.227 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.228 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10", 2024-05-27T08:48:23.228 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.228 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.228 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.228 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.228 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.228 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.229 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.229 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.229 INFO:teuthology.orchestra.run.smithi077.stdout: "mirror_snapshot_schedule": { 2024-05-27T08:48:23.229 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "mirror_snapshot_schedule", 2024-05-27T08:48:23.229 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.229 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.229 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.230 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.230 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.230 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.230 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.230 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.230 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.230 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.231 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.231 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.231 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.231 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.231 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.231 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.231 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.232 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.232 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.232 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.232 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.232 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.232 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.232 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.232 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.233 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.233 INFO:teuthology.orchestra.run.smithi077.stdout: "trash_purge_schedule": { 2024-05-27T08:48:23.233 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "trash_purge_schedule", 2024-05-27T08:48:23.233 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.233 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.233 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.233 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.234 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.234 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.234 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.234 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.234 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.234 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.235 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.235 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.235 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.235 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.235 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.235 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "restful", 2024-05-27T08:48:23.235 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.236 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.236 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.236 INFO:teuthology.orchestra.run.smithi077.stdout: "enable_auth": { 2024-05-27T08:48:23.236 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enable_auth", 2024-05-27T08:48:23.236 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.236 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.236 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.236 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.237 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.237 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.237 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.237 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.237 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.237 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.237 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.238 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.238 INFO:teuthology.orchestra.run.smithi077.stdout: "key_file": { 2024-05-27T08:48:23.238 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "key_file", 2024-05-27T08:48:23.238 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.238 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.238 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.239 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.239 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.239 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.239 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.239 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.239 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.239 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.240 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.240 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.240 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.240 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.240 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.240 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.240 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.240 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.241 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.241 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.241 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.241 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.241 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.241 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.242 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.242 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.242 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.242 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.242 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.242 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.242 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.243 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.243 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.243 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.243 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.243 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.243 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.243 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.243 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.244 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.244 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.244 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.244 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.244 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.244 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.244 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.245 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.245 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.245 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.245 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.245 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.245 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.245 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.246 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.246 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.246 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.246 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.246 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.246 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.246 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.247 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.247 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.247 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.247 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.247 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.247 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.247 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.247 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.248 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.248 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.248 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.248 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.248 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.248 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.248 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.249 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.249 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.249 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.249 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.249 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.249 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.249 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.250 INFO:teuthology.orchestra.run.smithi077.stdout: "server_addr": { 2024-05-27T08:48:23.250 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_addr", 2024-05-27T08:48:23.250 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.250 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.250 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.250 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.250 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.250 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.251 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.251 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.251 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.251 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.251 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.251 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.251 INFO:teuthology.orchestra.run.smithi077.stdout: "server_port": { 2024-05-27T08:48:23.252 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "server_port", 2024-05-27T08:48:23.252 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.252 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.252 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.252 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.252 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.252 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.253 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.253 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.253 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.253 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.253 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.253 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.253 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.254 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.254 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.254 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.254 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.254 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.254 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.254 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.255 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.255 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.255 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.255 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.255 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.255 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.255 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.255 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.256 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.256 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rgw", 2024-05-27T08:48:23.256 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.256 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.256 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.256 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.256 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.257 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.257 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.257 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.257 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.257 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.257 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.257 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.258 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.258 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.258 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.258 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.258 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.258 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.258 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.258 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.259 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.259 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.259 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.259 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.259 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.259 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.259 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.260 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.260 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.260 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.260 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.260 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.260 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.260 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.261 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.261 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.261 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.261 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.261 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.261 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.261 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.262 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.262 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.262 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.262 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.262 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.262 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.262 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.262 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.263 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.263 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.263 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.263 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.263 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.263 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.263 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.264 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.264 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.264 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.264 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.264 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.264 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.264 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.265 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.265 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.265 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.265 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.265 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.265 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.265 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.266 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.266 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.266 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.266 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.266 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.266 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.266 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.266 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.267 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.267 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.267 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.267 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.267 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.267 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.267 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.268 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.268 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.268 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.268 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.268 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.268 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rook", 2024-05-27T08:48:23.268 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.268 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.269 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.269 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.269 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.269 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.269 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.269 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.269 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.270 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.270 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.270 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.270 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.270 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.270 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.270 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.271 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.271 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.271 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.271 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.271 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.271 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.271 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.272 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.272 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.272 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.272 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.272 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.272 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.272 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.273 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.273 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.273 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.273 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.273 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.273 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.273 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.273 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.274 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.274 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.274 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.274 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.274 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.274 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.274 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.275 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.275 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.275 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.275 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.275 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.275 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.275 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.276 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.276 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.276 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.276 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.276 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.276 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.276 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.277 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.277 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.277 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.277 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.277 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.277 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.277 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.278 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.278 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.278 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.278 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.278 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.278 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.278 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.279 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.279 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.279 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.279 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.279 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.279 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.279 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.279 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.280 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.280 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.280 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.280 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.280 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.280 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.280 INFO:teuthology.orchestra.run.smithi077.stdout: "storage_class": { 2024-05-27T08:48:23.281 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "storage_class", 2024-05-27T08:48:23.281 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.281 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.281 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.281 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "local", 2024-05-27T08:48:23.281 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.281 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.282 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.282 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-27T08:48:23.282 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.282 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.282 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.282 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.282 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.283 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.283 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.283 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "selftest", 2024-05-27T08:48:23.283 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.283 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.283 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.283 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.283 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.284 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.284 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.284 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.284 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.284 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.284 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.284 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.285 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.285 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.285 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.285 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.285 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.285 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.285 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.285 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.286 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.286 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.286 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.286 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.286 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.286 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.286 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.287 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.287 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.287 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.287 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.287 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.287 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.287 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.288 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.288 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.288 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.288 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.288 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.288 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.288 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.289 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.289 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.289 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.289 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.289 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.289 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.289 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.289 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.290 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.291 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.291 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.291 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.291 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.291 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.291 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.291 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.292 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.292 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.292 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.292 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.292 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.292 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.292 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.292 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.293 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.293 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.293 INFO:teuthology.orchestra.run.smithi077.stdout: "roption1": { 2024-05-27T08:48:23.293 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption1", 2024-05-27T08:48:23.293 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.293 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.293 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.294 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.294 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.294 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.294 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.294 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.294 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.294 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.295 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.295 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.295 INFO:teuthology.orchestra.run.smithi077.stdout: "roption2": { 2024-05-27T08:48:23.295 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "roption2", 2024-05-27T08:48:23.295 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.295 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.295 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.295 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "xyz", 2024-05-27T08:48:23.296 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.296 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.296 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.296 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.296 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.296 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.296 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.297 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.297 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption1": { 2024-05-27T08:48:23.297 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption1", 2024-05-27T08:48:23.297 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.297 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.297 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.297 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.298 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.298 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.298 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.298 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.298 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.298 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.298 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.299 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.299 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption2": { 2024-05-27T08:48:23.299 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption2", 2024-05-27T08:48:23.299 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.299 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.299 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.299 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.299 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.300 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.300 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.300 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.300 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.300 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.300 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.300 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.301 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption3": { 2024-05-27T08:48:23.301 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption3", 2024-05-27T08:48:23.301 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "float", 2024-05-27T08:48:23.301 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.301 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.301 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.301 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.302 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.302 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.302 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.302 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.302 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.302 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.302 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.303 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption4": { 2024-05-27T08:48:23.303 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption4", 2024-05-27T08:48:23.303 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.303 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.303 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.303 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.303 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.304 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.304 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.304 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.304 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.304 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.304 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.304 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.305 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption5": { 2024-05-27T08:48:23.305 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption5", 2024-05-27T08:48:23.305 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.305 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.305 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.305 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.305 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.306 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.306 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.306 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.306 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.306 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.306 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.306 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.306 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption6": { 2024-05-27T08:48:23.307 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption6", 2024-05-27T08:48:23.307 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.307 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.307 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.307 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.307 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.307 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.308 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.308 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.308 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.308 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.308 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.308 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.308 INFO:teuthology.orchestra.run.smithi077.stdout: "rwoption7": { 2024-05-27T08:48:23.309 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rwoption7", 2024-05-27T08:48:23.309 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.309 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.309 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.309 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.309 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "1", 2024-05-27T08:48:23.309 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "42", 2024-05-27T08:48:23.309 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.310 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.310 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.310 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.310 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.310 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.310 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.310 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.311 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.311 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.311 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.311 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.311 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.311 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.311 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.312 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.312 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.312 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.312 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.312 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.312 INFO:teuthology.orchestra.run.smithi077.stdout: "testkey": { 2024-05-27T08:48:23.312 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testkey", 2024-05-27T08:48:23.313 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.313 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.313 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.313 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.313 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.313 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.313 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.313 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.314 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.314 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.314 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.314 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.314 INFO:teuthology.orchestra.run.smithi077.stdout: "testlkey": { 2024-05-27T08:48:23.314 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testlkey", 2024-05-27T08:48:23.314 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.315 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.315 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.315 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.315 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.315 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.315 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.315 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.316 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.316 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.316 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.316 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.316 INFO:teuthology.orchestra.run.smithi077.stdout: "testnewline": { 2024-05-27T08:48:23.316 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "testnewline", 2024-05-27T08:48:23.316 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.316 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.317 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.317 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.317 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.317 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.317 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.317 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.317 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.318 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.318 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.318 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.318 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.318 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.318 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.318 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "smb", 2024-05-27T08:48:23.319 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.319 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.319 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.319 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.319 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.319 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.319 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.319 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.320 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.320 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.320 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.320 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.320 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.320 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.320 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.321 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.321 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.321 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.321 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.321 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.321 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.321 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.322 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.322 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.322 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.322 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.322 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.322 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.322 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.323 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.323 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.323 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.323 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.323 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.323 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.323 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.324 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.324 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.324 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.324 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.324 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.324 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.324 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.324 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.325 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.325 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.325 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.325 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.325 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.325 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.325 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.326 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.326 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.326 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.326 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.326 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.326 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.326 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.327 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.327 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.327 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.327 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.327 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.327 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.327 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.328 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.328 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.328 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.328 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.328 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.328 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.328 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.329 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.329 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.329 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.329 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.329 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.329 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.329 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.330 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.330 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.330 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.330 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.330 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.330 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.330 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.331 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.331 INFO:teuthology.orchestra.run.smithi077.stdout: "update_orchestration": { 2024-05-27T08:48:23.331 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "update_orchestration", 2024-05-27T08:48:23.331 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.331 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.331 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.331 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.332 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.332 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.332 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.332 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-27T08:48:23.332 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.332 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.332 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.332 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.333 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.333 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.333 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.333 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-05-27T08:48:23.333 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.333 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.333 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.334 INFO:teuthology.orchestra.run.smithi077.stdout: "allow_m_granularity": { 2024-05-27T08:48:23.334 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "allow_m_granularity", 2024-05-27T08:48:23.334 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.334 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.334 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.334 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.334 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.335 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.335 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.335 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "allow minute scheduled snapshots", 2024-05-27T08:48:23.335 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.335 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.335 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.335 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.336 INFO:teuthology.orchestra.run.smithi077.stdout: "dump_on_update": { 2024-05-27T08:48:23.336 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "dump_on_update", 2024-05-27T08:48:23.336 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.336 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.336 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.336 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.336 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.337 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.337 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.337 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "dump database to debug log on update", 2024-05-27T08:48:23.337 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.337 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.337 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.337 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.337 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.338 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.338 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.338 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.338 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.338 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.338 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.339 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.339 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.339 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.339 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.339 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.339 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.339 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.339 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.340 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.340 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.340 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.340 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.340 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.340 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.340 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.341 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.341 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.341 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.341 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.341 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.341 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.341 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.341 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.342 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.342 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.342 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.342 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.342 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.342 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.343 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.343 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.343 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.343 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.343 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.343 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.343 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.344 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.344 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.344 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.344 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.344 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.344 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.344 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.344 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.345 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.345 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.345 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.345 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.345 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.345 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.345 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.346 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.346 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.346 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.346 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.346 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.346 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.346 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.347 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.347 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.347 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.347 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.347 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.347 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.347 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.348 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.348 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.348 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.348 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.348 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.348 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.348 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.349 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.349 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.349 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.349 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.349 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.349 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.349 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.349 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "stats", 2024-05-27T08:48:23.350 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.350 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.350 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.350 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.350 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.350 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.351 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.351 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.351 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.351 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.351 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.351 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.351 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.352 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.352 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.352 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.352 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.352 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.352 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.352 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.352 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.353 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.353 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.353 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.353 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.353 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.353 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.353 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.354 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.354 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.354 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.354 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.354 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.354 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.354 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.355 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.355 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.355 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.355 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.355 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.355 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.355 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.356 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.356 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.356 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.356 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.356 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.356 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.356 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.356 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.357 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.357 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.357 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.357 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.357 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.357 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.357 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.358 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.358 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.358 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.358 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.358 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.358 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.358 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.359 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.359 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.359 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.359 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.359 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.359 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.359 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.359 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.360 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.360 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.360 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.360 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.360 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.360 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.360 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.361 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.361 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.361 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.361 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.361 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.361 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.361 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.362 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.362 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.362 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.362 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.362 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "status", 2024-05-27T08:48:23.362 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.362 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.363 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.364 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.364 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.364 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.364 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.364 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.364 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.364 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.365 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.365 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.365 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.365 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.365 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.365 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.365 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.366 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.366 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.366 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.366 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.366 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.366 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.366 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.367 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.367 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.367 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.367 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.367 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.367 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.367 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.368 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.368 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.368 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.368 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.368 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.368 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.368 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.368 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.369 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.369 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.369 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.369 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.369 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.369 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.369 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.370 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.370 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.370 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.370 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.370 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.370 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.370 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.371 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.371 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.371 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.371 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.371 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.371 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.371 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.372 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.372 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.372 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.372 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.372 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.372 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.372 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.373 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.373 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.373 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.373 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.373 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.373 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.373 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.374 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.374 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.374 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.374 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.374 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.374 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.374 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.375 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.375 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.375 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telegraf", 2024-05-27T08:48:23.375 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.375 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.375 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.375 INFO:teuthology.orchestra.run.smithi077.stdout: "address": { 2024-05-27T08:48:23.376 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "address", 2024-05-27T08:48:23.376 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.376 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.376 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.376 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-27T08:48:23.376 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.376 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:23.377 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:23.378 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:23.378 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.378 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.378 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "15", 2024-05-27T08:48:23.378 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.378 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.378 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.379 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.380 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.380 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.380 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.380 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.380 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.380 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.380 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.381 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.381 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.381 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.381 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.381 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.381 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.381 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.382 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.382 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.382 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.382 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.382 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.382 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.382 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.383 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.383 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.383 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.383 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.383 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.383 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.383 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.383 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.384 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.384 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.384 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.384 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.384 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.384 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.384 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.385 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.385 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.385 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.385 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.385 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.385 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.385 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.386 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.386 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.386 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.386 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.386 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.386 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.386 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.387 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.388 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.388 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.388 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.388 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.388 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.388 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.388 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.388 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.389 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.389 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.389 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.389 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.389 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.389 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.389 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.390 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.390 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.390 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.390 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.390 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.390 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.390 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.391 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.391 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.391 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.391 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.391 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.391 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "telemetry", 2024-05-27T08:48:23.391 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.391 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.392 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.392 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_basic": { 2024-05-27T08:48:23.392 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_basic", 2024-05-27T08:48:23.392 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.392 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.392 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.392 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.393 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.393 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.393 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.393 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-27T08:48:23.393 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.393 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.393 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.394 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_crash": { 2024-05-27T08:48:23.394 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_crash", 2024-05-27T08:48:23.394 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.394 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.394 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.394 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.394 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.395 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.395 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.395 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-27T08:48:23.395 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.395 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.395 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.395 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.396 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_device": { 2024-05-27T08:48:23.396 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_device", 2024-05-27T08:48:23.396 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.396 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.396 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.396 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.396 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.397 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.397 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.397 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-27T08:48:23.397 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.397 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.397 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.397 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_ident": { 2024-05-27T08:48:23.398 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_ident", 2024-05-27T08:48:23.398 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.398 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.398 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.398 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.398 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.398 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.399 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.399 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-27T08:48:23.399 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.399 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.399 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.399 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.399 INFO:teuthology.orchestra.run.smithi077.stdout: "channel_perf": { 2024-05-27T08:48:23.400 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "channel_perf", 2024-05-27T08:48:23.400 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.400 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.400 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.400 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.400 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.400 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.401 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.401 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-27T08:48:23.401 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.401 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.401 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.401 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.401 INFO:teuthology.orchestra.run.smithi077.stdout: "contact": { 2024-05-27T08:48:23.402 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "contact", 2024-05-27T08:48:23.402 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.402 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.402 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.402 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.402 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.402 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.403 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.403 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.403 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.403 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.403 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.403 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.403 INFO:teuthology.orchestra.run.smithi077.stdout: "description": { 2024-05-27T08:48:23.403 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "description", 2024-05-27T08:48:23.404 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.404 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.404 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.404 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.404 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.404 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.404 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.405 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.405 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.405 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.405 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.405 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.405 INFO:teuthology.orchestra.run.smithi077.stdout: "device_url": { 2024-05-27T08:48:23.405 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "device_url", 2024-05-27T08:48:23.406 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.406 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.406 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.406 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-27T08:48:23.406 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.406 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.406 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.407 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.407 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.407 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.407 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.407 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.407 INFO:teuthology.orchestra.run.smithi077.stdout: "enabled": { 2024-05-27T08:48:23.407 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "enabled", 2024-05-27T08:48:23.407 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.408 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.408 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.408 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.408 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.408 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.408 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.408 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.409 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.409 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.409 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.409 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.409 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:23.409 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:23.409 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.410 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.410 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.410 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "24", 2024-05-27T08:48:23.410 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "8", 2024-05-27T08:48:23.410 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.410 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.410 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.411 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.411 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.411 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.411 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.411 INFO:teuthology.orchestra.run.smithi077.stdout: "last_opt_revision": { 2024-05-27T08:48:23.411 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "last_opt_revision", 2024-05-27T08:48:23.411 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.412 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.412 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.412 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "1", 2024-05-27T08:48:23.412 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.412 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.412 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.412 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.412 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard": { 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard", 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.413 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.414 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.414 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.414 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.414 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.414 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.414 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.414 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.415 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.415 INFO:teuthology.orchestra.run.smithi077.stdout: "leaderboard_description": { 2024-05-27T08:48:23.415 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "leaderboard_description", 2024-05-27T08:48:23.415 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.415 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.415 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.415 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.416 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.416 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.416 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.416 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.416 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.416 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.416 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.417 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.417 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.417 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.417 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.417 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.417 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.417 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.418 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.418 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.418 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.418 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.418 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.418 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.418 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.419 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.419 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.419 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.419 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.419 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.419 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.419 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.420 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.420 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.420 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.420 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.420 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.420 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.420 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.421 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.421 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.421 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.421 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.421 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.421 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.421 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.422 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.422 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.422 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.422 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.422 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.422 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.422 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.423 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.423 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.423 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.423 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.423 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.423 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.423 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.423 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.424 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.424 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.424 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.424 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.424 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.424 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.424 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.425 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.425 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.425 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.425 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.425 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.425 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.425 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.426 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.426 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.426 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.426 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.426 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.426 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.426 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.426 INFO:teuthology.orchestra.run.smithi077.stdout: "organization": { 2024-05-27T08:48:23.427 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "organization", 2024-05-27T08:48:23.427 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.427 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.427 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.427 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.427 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.427 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.428 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.428 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.428 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.428 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.428 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.428 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.428 INFO:teuthology.orchestra.run.smithi077.stdout: "proxy": { 2024-05-27T08:48:23.429 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "proxy", 2024-05-27T08:48:23.429 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.429 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.429 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.429 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.429 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.429 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.429 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.430 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.430 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.430 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.430 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.430 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.430 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.430 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.431 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.431 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.431 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.431 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.431 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.431 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.431 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.432 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.432 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.432 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.432 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.432 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.432 INFO:teuthology.orchestra.run.smithi077.stdout: "url": { 2024-05-27T08:48:23.432 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "url", 2024-05-27T08:48:23.432 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.433 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.433 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.433 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-27T08:48:23.433 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.433 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.433 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.433 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.434 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.434 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.434 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.434 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.434 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.434 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.434 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "test_orchestrator", 2024-05-27T08:48:23.435 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.435 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.435 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.435 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.435 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.435 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.436 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.436 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.436 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.436 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.436 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.436 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.436 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.437 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.437 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.437 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.437 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.437 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.437 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.437 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.438 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.438 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.438 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.438 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.438 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.438 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.439 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.439 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.439 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.439 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.439 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.439 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.439 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.440 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.440 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.440 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.440 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.440 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.440 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.440 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.440 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.441 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.441 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.441 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.441 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.441 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.441 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.441 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.442 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.442 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.442 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.442 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.442 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.442 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.442 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.443 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.443 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.443 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.443 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.443 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.443 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.444 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.444 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.444 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.444 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.444 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.444 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.444 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.445 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.445 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.445 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.445 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.445 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.445 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.446 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.446 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.446 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.446 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.446 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.446 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.446 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.447 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.447 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.447 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.447 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.447 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.447 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.447 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-05-27T08:48:23.448 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.448 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.448 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.448 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.448 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.448 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.448 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.449 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.449 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.449 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.449 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.449 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.449 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.449 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.450 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.450 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.450 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.450 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.450 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.450 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.450 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.450 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.451 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.452 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.452 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.452 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.452 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.452 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.452 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.452 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.453 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.453 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.453 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.453 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.453 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.453 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.453 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.454 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.454 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.454 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.454 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.454 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.454 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.454 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.455 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.455 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.455 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.455 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.455 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.455 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.455 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.455 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.456 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.456 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.456 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.456 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.456 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.456 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.456 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.457 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.457 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.457 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.457 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.457 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.457 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.457 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.458 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.458 INFO:teuthology.orchestra.run.smithi077.stdout: "max_concurrent_clones": { 2024-05-27T08:48:23.458 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "max_concurrent_clones", 2024-05-27T08:48:23.458 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.458 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.458 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.458 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "4", 2024-05-27T08:48:23.459 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.459 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.459 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.459 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-27T08:48:23.459 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.459 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.459 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.460 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.460 INFO:teuthology.orchestra.run.smithi077.stdout: "periodic_async_work": { 2024-05-27T08:48:23.460 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "periodic_async_work", 2024-05-27T08:48:23.460 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.460 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.460 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.460 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.461 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.461 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.461 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.461 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Periodically check for async work", 2024-05-27T08:48:23.461 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.461 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.461 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.461 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.462 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_delay": { 2024-05-27T08:48:23.462 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_delay", 2024-05-27T08:48:23.462 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.462 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.462 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.462 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.463 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.463 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.463 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.463 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-27T08:48:23.463 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.463 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.463 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.463 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.464 INFO:teuthology.orchestra.run.smithi077.stdout: "snapshot_clone_no_wait": { 2024-05-27T08:48:23.464 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snapshot_clone_no_wait", 2024-05-27T08:48:23.464 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.464 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.464 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.464 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "True", 2024-05-27T08:48:23.464 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.465 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.465 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.465 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-27T08:48:23.465 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.465 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.465 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.465 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.465 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.466 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.466 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.466 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.466 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.466 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.466 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.466 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.467 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.467 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.467 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.467 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.467 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.467 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.467 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.468 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.468 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.468 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix", 2024-05-27T08:48:23.468 INFO:teuthology.orchestra.run.smithi077.stdout: "can_run": true, 2024-05-27T08:48:23.468 INFO:teuthology.orchestra.run.smithi077.stdout: "error_string": "", 2024-05-27T08:48:23.468 INFO:teuthology.orchestra.run.smithi077.stdout: "module_options": { 2024-05-27T08:48:23.468 INFO:teuthology.orchestra.run.smithi077.stdout: "discovery_interval": { 2024-05-27T08:48:23.469 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "discovery_interval", 2024-05-27T08:48:23.469 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "uint", 2024-05-27T08:48:23.469 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.469 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.469 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "100", 2024-05-27T08:48:23.469 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.469 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.470 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.470 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.470 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.470 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.470 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.470 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.470 INFO:teuthology.orchestra.run.smithi077.stdout: "identifier": { 2024-05-27T08:48:23.471 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "identifier", 2024-05-27T08:48:23.471 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.471 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.471 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.471 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.471 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.471 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.472 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.472 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.472 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.472 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.472 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.472 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.472 INFO:teuthology.orchestra.run.smithi077.stdout: "interval": { 2024-05-27T08:48:23.473 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "interval", 2024-05-27T08:48:23.473 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "secs", 2024-05-27T08:48:23.473 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.473 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.473 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "60", 2024-05-27T08:48:23.473 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.473 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.474 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.474 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.474 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.474 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.474 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.474 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.474 INFO:teuthology.orchestra.run.smithi077.stdout: "log_level": { 2024-05-27T08:48:23.475 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_level", 2024-05-27T08:48:23.475 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.475 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.475 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.475 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.475 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.475 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.475 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.476 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.476 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.476 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.476 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.476 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.476 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.476 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.477 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.477 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.477 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.477 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.477 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.477 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster": { 2024-05-27T08:48:23.477 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster", 2024-05-27T08:48:23.477 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.478 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.478 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.478 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.478 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.478 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.478 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.478 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.479 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.479 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.479 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.479 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.479 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_cluster_level": { 2024-05-27T08:48:23.479 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_cluster_level", 2024-05-27T08:48:23.479 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.480 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.480 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.480 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "info", 2024-05-27T08:48:23.480 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.480 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.480 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [ 2024-05-27T08:48:23.480 INFO:teuthology.orchestra.run.smithi077.stdout: "", 2024-05-27T08:48:23.480 INFO:teuthology.orchestra.run.smithi077.stdout: "critical", 2024-05-27T08:48:23.481 INFO:teuthology.orchestra.run.smithi077.stdout: "debug", 2024-05-27T08:48:23.481 INFO:teuthology.orchestra.run.smithi077.stdout: "error", 2024-05-27T08:48:23.481 INFO:teuthology.orchestra.run.smithi077.stdout: "info", 2024-05-27T08:48:23.481 INFO:teuthology.orchestra.run.smithi077.stdout: "warning" 2024-05-27T08:48:23.481 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.481 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.481 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.482 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.482 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.482 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.482 INFO:teuthology.orchestra.run.smithi077.stdout: "log_to_file": { 2024-05-27T08:48:23.482 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "log_to_file", 2024-05-27T08:48:23.482 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "bool", 2024-05-27T08:48:23.482 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.482 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.483 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "False", 2024-05-27T08:48:23.483 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.483 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.483 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.483 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.483 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.484 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.484 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.484 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.484 INFO:teuthology.orchestra.run.smithi077.stdout: "sqlite3_killpoint": { 2024-05-27T08:48:23.484 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "sqlite3_killpoint", 2024-05-27T08:48:23.484 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.484 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "dev", 2024-05-27T08:48:23.485 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 1, 2024-05-27T08:48:23.485 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "0", 2024-05-27T08:48:23.485 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.485 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.485 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.485 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.485 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.486 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.486 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.486 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.486 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_host": { 2024-05-27T08:48:23.486 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_host", 2024-05-27T08:48:23.486 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.486 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.487 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.487 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "", 2024-05-27T08:48:23.487 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.487 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.487 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.487 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.487 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.487 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.488 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.488 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.488 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_port": { 2024-05-27T08:48:23.488 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_port", 2024-05-27T08:48:23.488 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "int", 2024-05-27T08:48:23.488 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "10051", 2024-05-27T08:48:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.489 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.490 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.490 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.490 INFO:teuthology.orchestra.run.smithi077.stdout: "zabbix_sender": { 2024-05-27T08:48:23.490 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "zabbix_sender", 2024-05-27T08:48:23.490 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "str", 2024-05-27T08:48:23.490 INFO:teuthology.orchestra.run.smithi077.stdout: "level": "advanced", 2024-05-27T08:48:23.490 INFO:teuthology.orchestra.run.smithi077.stdout: "flags": 0, 2024-05-27T08:48:23.491 INFO:teuthology.orchestra.run.smithi077.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-27T08:48:23.491 INFO:teuthology.orchestra.run.smithi077.stdout: "min": "", 2024-05-27T08:48:23.491 INFO:teuthology.orchestra.run.smithi077.stdout: "max": "", 2024-05-27T08:48:23.491 INFO:teuthology.orchestra.run.smithi077.stdout: "enum_allowed": [], 2024-05-27T08:48:23.491 INFO:teuthology.orchestra.run.smithi077.stdout: "desc": "", 2024-05-27T08:48:23.491 INFO:teuthology.orchestra.run.smithi077.stdout: "long_desc": "", 2024-05-27T08:48:23.491 INFO:teuthology.orchestra.run.smithi077.stdout: "tags": [], 2024-05-27T08:48:23.492 INFO:teuthology.orchestra.run.smithi077.stdout: "see_also": [] 2024-05-27T08:48:23.492 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.492 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.492 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.492 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.492 INFO:teuthology.orchestra.run.smithi077.stdout: "services": { 2024-05-27T08:48:23.492 INFO:teuthology.orchestra.run.smithi077.stdout: "dashboard": "https://172.21.15.77:8443/" 2024-05-27T08:48:23.493 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.493 INFO:teuthology.orchestra.run.smithi077.stdout: "always_on_modules": { 2024-05-27T08:48:23.493 INFO:teuthology.orchestra.run.smithi077.stdout: "octopus": [ 2024-05-27T08:48:23.493 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-05-27T08:48:23.493 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-05-27T08:48:23.493 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-05-27T08:48:23.493 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-05-27T08:48:23.494 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-05-27T08:48:23.494 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-05-27T08:48:23.494 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-05-27T08:48:23.494 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-05-27T08:48:23.494 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-05-27T08:48:23.494 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-05-27T08:48:23.494 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.495 INFO:teuthology.orchestra.run.smithi077.stdout: "pacific": [ 2024-05-27T08:48:23.495 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-05-27T08:48:23.495 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-05-27T08:48:23.495 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-05-27T08:48:23.495 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-05-27T08:48:23.495 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-05-27T08:48:23.495 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-05-27T08:48:23.496 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-05-27T08:48:23.496 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-05-27T08:48:23.496 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-05-27T08:48:23.496 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-05-27T08:48:23.496 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.496 INFO:teuthology.orchestra.run.smithi077.stdout: "quincy": [ 2024-05-27T08:48:23.496 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-05-27T08:48:23.497 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-05-27T08:48:23.497 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-05-27T08:48:23.497 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-05-27T08:48:23.497 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-05-27T08:48:23.497 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-05-27T08:48:23.497 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-05-27T08:48:23.497 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-05-27T08:48:23.497 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-05-27T08:48:23.498 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-05-27T08:48:23.498 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.498 INFO:teuthology.orchestra.run.smithi077.stdout: "reef": [ 2024-05-27T08:48:23.498 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-05-27T08:48:23.498 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-05-27T08:48:23.498 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-05-27T08:48:23.498 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-05-27T08:48:23.499 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-05-27T08:48:23.499 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-05-27T08:48:23.499 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-05-27T08:48:23.499 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-05-27T08:48:23.499 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-05-27T08:48:23.499 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-05-27T08:48:23.499 INFO:teuthology.orchestra.run.smithi077.stdout: ], 2024-05-27T08:48:23.499 INFO:teuthology.orchestra.run.smithi077.stdout: "squid": [ 2024-05-27T08:48:23.500 INFO:teuthology.orchestra.run.smithi077.stdout: "balancer", 2024-05-27T08:48:23.500 INFO:teuthology.orchestra.run.smithi077.stdout: "crash", 2024-05-27T08:48:23.500 INFO:teuthology.orchestra.run.smithi077.stdout: "devicehealth", 2024-05-27T08:48:23.500 INFO:teuthology.orchestra.run.smithi077.stdout: "orchestrator", 2024-05-27T08:48:23.500 INFO:teuthology.orchestra.run.smithi077.stdout: "pg_autoscaler", 2024-05-27T08:48:23.500 INFO:teuthology.orchestra.run.smithi077.stdout: "progress", 2024-05-27T08:48:23.500 INFO:teuthology.orchestra.run.smithi077.stdout: "rbd_support", 2024-05-27T08:48:23.501 INFO:teuthology.orchestra.run.smithi077.stdout: "status", 2024-05-27T08:48:23.501 INFO:teuthology.orchestra.run.smithi077.stdout: "telemetry", 2024-05-27T08:48:23.501 INFO:teuthology.orchestra.run.smithi077.stdout: "volumes" 2024-05-27T08:48:23.501 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:23.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.501 INFO:teuthology.orchestra.run.smithi077.stdout: "last_failure_osd_epoch": 86, 2024-05-27T08:48:23.501 INFO:teuthology.orchestra.run.smithi077.stdout: "active_clients": [ 2024-05-27T08:48:23.501 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.502 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "devicehealth", 2024-05-27T08:48:23.502 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-05-27T08:48:23.502 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.502 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-05-27T08:48:23.502 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-05-27T08:48:23.502 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 1145349718 2024-05-27T08:48:23.502 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.503 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:23.503 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.503 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.503 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "libcephsqlite", 2024-05-27T08:48:23.503 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-05-27T08:48:23.503 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.503 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-05-27T08:48:23.504 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-05-27T08:48:23.504 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 4095536442 2024-05-27T08:48:23.504 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.504 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:23.504 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.504 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.504 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "rbd_support", 2024-05-27T08:48:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-05-27T08:48:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-05-27T08:48:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-05-27T08:48:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3036574541 2024-05-27T08:48:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:23.505 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.506 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.506 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-05-27T08:48:23.506 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-05-27T08:48:23.506 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.506 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-05-27T08:48:23.506 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-05-27T08:48:23.506 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 2123681859 2024-05-27T08:48:23.507 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.507 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:23.507 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.507 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.507 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "snap_schedule", 2024-05-27T08:48:23.507 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-05-27T08:48:23.507 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.508 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-05-27T08:48:23.508 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-05-27T08:48:23.508 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 2633646745 2024-05-27T08:48:23.508 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.508 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:23.508 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:23.508 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.509 INFO:teuthology.orchestra.run.smithi077.stdout: "name": "volumes", 2024-05-27T08:48:23.509 INFO:teuthology.orchestra.run.smithi077.stdout: "addrvec": [ 2024-05-27T08:48:23.509 INFO:teuthology.orchestra.run.smithi077.stdout: { 2024-05-27T08:48:23.509 INFO:teuthology.orchestra.run.smithi077.stdout: "type": "v2", 2024-05-27T08:48:23.509 INFO:teuthology.orchestra.run.smithi077.stdout: "addr": "172.21.15.77:0", 2024-05-27T08:48:23.509 INFO:teuthology.orchestra.run.smithi077.stdout: "nonce": 3334207125 2024-05-27T08:48:23.509 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.509 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:23.510 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:23.510 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-05-27T08:48:23.510 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:48:23.511 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:21 smithi092 ceph-mon[36049]: pgmap v313: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 684 KiB/s wr, 45 op/s 2024-05-27T08:48:23.511 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:21 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3854693004' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:48:23.512 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:20 smithi179 ceph-mon[36481]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:48:23.512 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:20 smithi179 ceph-mon[36481]: from='client.15807 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:48:23.512 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:20 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/846132353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:48:23.512 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:20 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2136843291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:48:23.512 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:20 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1571968714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:48:23.513 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:20 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2629183314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:48:23.513 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:21 smithi179 ceph-mon[36481]: pgmap v313: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 684 KiB/s wr, 45 op/s 2024-05-27T08:48:23.513 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:21 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3854693004' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:48:23.513 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f20840b2960 msgr2=0x7f20840b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:23.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f20840b2960 0x7f20840b4e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f208c000c00 tx=0x7f208c032ca0 comp rx=0 tx=0).stop 2024-05-27T08:48:23.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 msgr2=0x7f209c19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:23.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 0x7f209c19db40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f209400cad0 tx=0x7f209400cf90 comp rx=0 tx=0).stop 2024-05-27T08:48:23.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 shutdown_connections 2024-05-27T08:48:23.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f20840b2960 0x7f20840b4e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:23.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f209c1033f0 0x7f209c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:23.514 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f209c108a40 0x7f209c19db40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:23.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 --2- 172.21.15.77:0/3854693004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f209c102a30 0x7f209c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:23.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 >> 172.21.15.77:0/3854693004 conn(0x7f209c079e50 msgr2=0x7f209c105640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:23.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 shutdown_connections 2024-05-27T08:48:23.515 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:21.012+0000 7f20a24fc640 1 -- 172.21.15.77:0/3854693004 wait complete. 2024-05-27T08:48:23.515 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:20 smithi077 ceph-mon[32007]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:48:23.516 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:20 smithi077 ceph-mon[32007]: from='client.15807 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-27T08:48:23.516 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:20 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/846132353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:48:23.516 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:20 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2136843291' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-27T08:48:23.516 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:20 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1571968714' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-27T08:48:23.516 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:20 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2629183314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-27T08:48:23.525 DEBUG:tasks.check_counter:Getting stats from a 2024-05-27T08:48:23.525 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-27T08:48:23.526 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:21 smithi077 ceph-mon[32007]: pgmap v313: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 684 KiB/s wr, 45 op/s 2024-05-27T08:48:23.526 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:21 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3854693004' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-27T08:48:23.783 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:48:23.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:23 smithi077 ceph-mon[32007]: pgmap v314: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 855 KiB/s wr, 58 op/s 2024-05-27T08:48:23.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:23 smithi092 ceph-mon[36049]: pgmap v314: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 855 KiB/s wr, 58 op/s 2024-05-27T08:48:24.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:23 smithi179 ceph-mon[36481]: pgmap v314: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 855 KiB/s wr, 58 op/s 2024-05-27T08:48:25.491 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:48:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:25.491 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1042, 2024-05-27T08:48:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1017, 2024-05-27T08:48:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 2081488, 2024-05-27T08:48:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 385720, 2024-05-27T08:48:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-05-27T08:48:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.128319756, 2024-05-27T08:48:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.041167571, 2024-05-27T08:48:25.492 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.060756153, 2024-05-27T08:48:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.029723038, 2024-05-27T08:48:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1017, 2024-05-27T08:48:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.045097966, 2024-05-27T08:48:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000044344 2024-05-27T08:48:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:25.493 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1819408, 2024-05-27T08:48:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 137680 2024-05-27T08:48:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.494 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 693, 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 858, 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 4367358, 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 675630, 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 7, 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.075102232, 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.025104524, 2024-05-27T08:48:25.495 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.020181968, 2024-05-27T08:48:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.022657058, 2024-05-27T08:48:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 855, 2024-05-27T08:48:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.032021131, 2024-05-27T08:48:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000037451 2024-05-27T08:48:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:25.496 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T08:48:25.497 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 455824 2024-05-27T08:48:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 932, 2024-05-27T08:48:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 934, 2024-05-27T08:48:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 6478399, 2024-05-27T08:48:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 296377, 2024-05-27T08:48:25.498 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-05-27T08:48:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.077305621, 2024-05-27T08:48:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.023959791, 2024-05-27T08:48:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.024579770, 2024-05-27T08:48:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.022522362, 2024-05-27T08:48:25.499 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 932, 2024-05-27T08:48:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.026735786, 2024-05-27T08:48:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000028686 2024-05-27T08:48:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:25.500 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:48:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:48:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.501 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-05-27T08:48:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-05-27T08:48:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-05-27T08:48:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-05-27T08:48:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-27T08:48:25.502 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-27T08:48:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4055, 2024-05-27T08:48:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.100943245, 2024-05-27T08:48:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000024893 2024-05-27T08:48:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.503 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-27T08:48:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-27T08:48:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.504 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-05-27T08:48:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-05-27T08:48:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-05-27T08:48:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-05-27T08:48:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.505 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-05-27T08:48:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-05-27T08:48:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:25.506 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-05-27T08:48:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-05-27T08:48:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-05-27T08:48:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 994, 2024-05-27T08:48:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 28, 2024-05-27T08:48:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 966, 2024-05-27T08:48:25.507 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 954, 2024-05-27T08:48:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-05-27T08:48:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-05-27T08:48:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-05-27T08:48:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 6, 2024-05-27T08:48:25.508 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-05-27T08:48:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-05-27T08:48:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-05-27T08:48:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-05-27T08:48:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:25.509 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-05-27T08:48:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-05-27T08:48:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-05-27T08:48:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-05-27T08:48:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-05-27T08:48:25.510 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-05-27T08:48:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-05-27T08:48:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2024-05-27T08:48:25.511 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2024-05-27T08:48:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-05-27T08:48:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:25.512 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:48:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T08:48:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-05-27T08:48:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:25.513 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-05-27T08:48:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-05-27T08:48:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-05-27T08:48:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-05-27T08:48:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-05-27T08:48:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-05-27T08:48:25.514 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-05-27T08:48:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-05-27T08:48:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-05-27T08:48:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-05-27T08:48:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-05-27T08:48:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-05-27T08:48:25.515 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-05-27T08:48:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-05-27T08:48:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-05-27T08:48:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-05-27T08:48:25.516 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:48:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:48:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-05-27T08:48:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:25.517 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-05-27T08:48:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-05-27T08:48:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-05-27T08:48:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-05-27T08:48:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-05-27T08:48:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-05-27T08:48:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 7290, 2024-05-27T08:48:25.518 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-05-27T08:48:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-05-27T08:48:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-05-27T08:48:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-05-27T08:48:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-05-27T08:48:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 10, 2024-05-27T08:48:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-05-27T08:48:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-05-27T08:48:25.519 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-05-27T08:48:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-05-27T08:48:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 14627054, 2024-05-27T08:48:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 14620709, 2024-05-27T08:48:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-05-27T08:48:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.520 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 7290 2024-05-27T08:48:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-05-27T08:48:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 195, 2024-05-27T08:48:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 983, 2024-05-27T08:48:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 788, 2024-05-27T08:48:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 24, 2024-05-27T08:48:25.521 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 24, 2024-05-27T08:48:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 0, 2024-05-27T08:48:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 994, 2024-05-27T08:48:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 994, 2024-05-27T08:48:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 0, 2024-05-27T08:48:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-05-27T08:48:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-05-27T08:48:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-05-27T08:48:25.522 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 53192, 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215416 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-05-27T08:48:25.523 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-05-27T08:48:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-05-27T08:48:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.524 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-05-27T08:48:25.525 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-05-27T08:48:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.526 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-05-27T08:48:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.527 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-05-27T08:48:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.528 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.529 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-05-27T08:48:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.530 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-05-27T08:48:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.531 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-05-27T08:48:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-05-27T08:48:25.532 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-05-27T08:48:25.533 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-05-27T08:48:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.534 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-05-27T08:48:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:25.535 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.536 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-05-27T08:48:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.537 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:25.538 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-05-27T08:48:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.539 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.540 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.541 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-05-27T08:48:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-05-27T08:48:25.542 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-05-27T08:48:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.543 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-05-27T08:48:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.544 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.545 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-05-27T08:48:25.546 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-05-27T08:48:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 3, 2024-05-27T08:48:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 1, 2024-05-27T08:48:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-05-27T08:48:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-05-27T08:48:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-05-27T08:48:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-05-27T08:48:25.547 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 2943, 2024-05-27T08:48:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-05-27T08:48:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:25.548 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:25.549 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:25.550 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:25.551 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-05-27T08:48:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 206135, 2024-05-27T08:48:25.552 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 250, 2024-05-27T08:48:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-05-27T08:48:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-05-27T08:48:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:25.553 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 169, 2024-05-27T08:48:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-05-27T08:48:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 1235685, 2024-05-27T08:48:25.554 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 13117, 2024-05-27T08:48:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-05-27T08:48:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-05-27T08:48:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-05-27T08:48:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-05-27T08:48:25.555 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 2403, 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 2403, 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2403, 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.344972714, 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000559705 2024-05-27T08:48:25.556 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-05-27T08:48:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-05-27T08:48:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2403, 2024-05-27T08:48:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2404 2024-05-27T08:48:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 2403, 2024-05-27T08:48:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 2403, 2024-05-27T08:48:25.557 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-05-27T08:48:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-05-27T08:48:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-05-27T08:48:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1388, 2024-05-27T08:48:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-05-27T08:48:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 1013, 2024-05-27T08:48:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-05-27T08:48:25.558 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-05-27T08:48:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-05-27T08:48:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-05-27T08:48:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-05-27T08:48:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-05-27T08:48:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-05-27T08:48:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-05-27T08:48:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:25.559 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2024-05-27T08:48:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-05-27T08:48:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-05-27T08:48:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-05-27T08:48:25.560 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-05-27T08:48:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-05-27T08:48:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-05-27T08:48:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-05-27T08:48:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-05-27T08:48:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-05-27T08:48:25.561 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-05-27T08:48:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-05-27T08:48:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-05-27T08:48:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-05-27T08:48:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-05-27T08:48:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-05-27T08:48:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-05-27T08:48:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-05-27T08:48:25.562 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 94, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 11, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 6, 2024-05-27T08:48:25.563 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 6, 2024-05-27T08:48:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-05-27T08:48:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-05-27T08:48:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-05-27T08:48:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-05-27T08:48:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-05-27T08:48:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-05-27T08:48:25.564 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-05-27T08:48:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-05-27T08:48:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-05-27T08:48:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-05-27T08:48:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-05-27T08:48:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.565 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-05-27T08:48:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T08:48:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-05-27T08:48:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-05-27T08:48:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-05-27T08:48:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-05-27T08:48:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:25.566 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-27T08:48:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 2663, 2024-05-27T08:48:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 12671367, 2024-05-27T08:48:25.567 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 2663, 2024-05-27T08:48:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-27T08:48:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-27T08:48:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2663, 2024-05-27T08:48:25.568 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 12671367, 2024-05-27T08:48:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:25.569 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-27T08:48:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-27T08:48:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-27T08:48:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2403, 2024-05-27T08:48:25.570 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 10432755, 2024-05-27T08:48:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2403, 2024-05-27T08:48:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 10432755, 2024-05-27T08:48:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.571 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-05-27T08:48:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-27T08:48:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-27T08:48:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:25.572 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2403, 2024-05-27T08:48:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 2403, 2024-05-27T08:48:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2403, 2024-05-27T08:48:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2403, 2024-05-27T08:48:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.573 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-27T08:48:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:25.574 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-27T08:48:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-27T08:48:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-27T08:48:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-27T08:48:25.575 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-05-27T08:48:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-05-27T08:48:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.576 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55891d9c86e0": { 2024-05-27T08:48:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-27T08:48:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-27T08:48:25.577 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-27T08:48:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-27T08:48:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-27T08:48:25.578 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-05-27T08:48:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-05-27T08:48:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:25.579 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.580 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:25.580 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:48:25.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:25 smithi077 ceph-mon[32007]: pgmap v315: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 545 KiB/s wr, 38 op/s 2024-05-27T08:48:25.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:25 smithi092 ceph-mon[36049]: pgmap v315: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 545 KiB/s wr, 38 op/s 2024-05-27T08:48:26.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:25 smithi179 ceph-mon[36481]: pgmap v315: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 545 KiB/s wr, 38 op/s 2024-05-27T08:48:26.356 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-27T08:48:26.356 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-27T08:48:26.357 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-05-27T08:48:26.357 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-05-27T08:48:26.357 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-27T08:48:26.357 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:26.357 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:26.357 DEBUG:tasks.check_counter:Getting stats from d 2024-05-27T08:48:26.357 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-27T08:48:26.604 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:48:27.834 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:27 smithi077 ceph-mon[32007]: pgmap v316: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 545 KiB/s wr, 38 op/s 2024-05-27T08:48:27.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:27 smithi092 ceph-mon[36049]: pgmap v316: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 545 KiB/s wr, 38 op/s 2024-05-27T08:48:28.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:27 smithi179 ceph-mon[36481]: pgmap v316: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 545 KiB/s wr, 38 op/s 2024-05-27T08:48:28.391 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:48:28.392 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:28.392 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-05-27T08:48:28.392 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-05-27T08:48:28.392 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-05-27T08:48:28.392 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-05-27T08:48:28.392 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-05-27T08:48:28.392 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.001245645, 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000318248, 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:28.393 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.394 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:28.394 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:28.394 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:28.394 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:28.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.394 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:48:28.394 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:48:28.394 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 2, 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 172, 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 320, 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 187664, 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.013775996, 2024-05-27T08:48:28.395 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.009518185, 2024-05-27T08:48:28.396 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001930739, 2024-05-27T08:48:28.396 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:48:28.396 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:28.396 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 169, 2024-05-27T08:48:28.396 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.017069056, 2024-05-27T08:48:28.396 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000101000 2024-05-27T08:48:28.396 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.396 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 187664 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:28.397 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 240, 2024-05-27T08:48:28.398 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 227, 2024-05-27T08:48:28.398 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1717648, 2024-05-27T08:48:28.398 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 138752, 2024-05-27T08:48:28.398 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-05-27T08:48:28.398 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-05-27T08:48:28.398 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.044753354, 2024-05-27T08:48:28.398 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.013719434, 2024-05-27T08:48:28.398 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.037463700, 2024-05-27T08:48:28.399 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.004280752, 2024-05-27T08:48:28.399 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:28.399 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 227, 2024-05-27T08:48:28.399 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.010323827, 2024-05-27T08:48:28.399 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000045479 2024-05-27T08:48:28.399 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.399 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:28.399 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1717648, 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 138752 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-05-27T08:48:28.400 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:28.401 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:28.402 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:28.402 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:28.402 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:28.402 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:28.402 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:28.402 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:28.402 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:28.403 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:28.403 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:28.403 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:28.403 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:28.403 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:28.403 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:28.403 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:28.403 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:28.404 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:28.405 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-05-27T08:48:28.405 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:28.405 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:28.405 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:28.405 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:28.405 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148748, 2024-05-27T08:48:28.405 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 32, 2024-05-27T08:48:28.405 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:28.406 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:28.406 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-05-27T08:48:28.406 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-05-27T08:48:28.406 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:28.406 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:28.406 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:28.407 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:28.407 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-05-27T08:48:28.407 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-05-27T08:48:28.407 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:28.407 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:28.407 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:28.407 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-05-27T08:48:28.408 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-05-27T08:48:28.408 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-05-27T08:48:28.408 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:28.408 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-05-27T08:48:28.408 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:28.408 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-05-27T08:48:28.408 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:28.408 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:28.409 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:28.409 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-27T08:48:28.409 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:28.409 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 238, 2024-05-27T08:48:28.409 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1681353, 2024-05-27T08:48:28.409 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:28.409 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 238, 2024-05-27T08:48:28.409 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-27T08:48:28.410 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-27T08:48:28.410 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 238, 2024-05-27T08:48:28.410 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1681353, 2024-05-27T08:48:28.410 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:28.410 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:28.410 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:28.410 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:28.411 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:28.411 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:28.411 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:48:29.013 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-27T08:48:29.013 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-27T08:48:29.013 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-27T08:48:29.014 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-27T08:48:29.014 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-27T08:48:29.014 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-27T08:48:29.014 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-27T08:48:29.014 DEBUG:tasks.check_counter:Getting stats from g 2024-05-27T08:48:29.014 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-27T08:48:29.255 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:48:29.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:29 smithi077 ceph-mon[32007]: pgmap v317: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 820 KiB/s wr, 56 op/s 2024-05-27T08:48:29.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:29 smithi092 ceph-mon[36049]: pgmap v317: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 820 KiB/s wr, 56 op/s 2024-05-27T08:48:30.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:29 smithi179 ceph-mon[36481]: pgmap v317: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 820 KiB/s wr, 56 op/s 2024-05-27T08:48:30.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:30 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:48:30.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:30 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:48:30.997 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:30 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:48:30.997 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:30 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:48:31.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:30 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:48:31.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:30 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-27T08:48:31.133 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:48:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1070, 2024-05-27T08:48:31.133 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1045, 2024-05-27T08:48:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 2093648, 2024-05-27T08:48:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 395778, 2024-05-27T08:48:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-05-27T08:48:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.125946304, 2024-05-27T08:48:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.048067725, 2024-05-27T08:48:31.134 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.056148702, 2024-05-27T08:48:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.025195228, 2024-05-27T08:48:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1044, 2024-05-27T08:48:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.046596057, 2024-05-27T08:48:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000044632 2024-05-27T08:48:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:31.135 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1823968, 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 139152 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1611, 2024-05-27T08:48:31.136 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1778, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 9424061, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 969511, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 7, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.113758199, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.033077162, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.036795069, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.034951386, 2024-05-27T08:48:31.137 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1775, 2024-05-27T08:48:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.049942628, 2024-05-27T08:48:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000028136 2024-05-27T08:48:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.138 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T08:48:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 457792 2024-05-27T08:48:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 8, 2024-05-27T08:48:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 8, 2024-05-27T08:48:31.139 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 4621, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 1732, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.001355877, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000087522, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001149985, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000078047, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 7, 2024-05-27T08:48:31.140 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002600497, 2024-05-27T08:48:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000371499 2024-05-27T08:48:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.141 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T08:48:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T08:48:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-05-27T08:48:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-05-27T08:48:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-05-27T08:48:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-05-27T08:48:31.142 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4177, 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.089947573, 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000021534 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-05-27T08:48:31.143 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-05-27T08:48:31.144 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-05-27T08:48:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-05-27T08:48:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-05-27T08:48:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:31.145 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-05-27T08:48:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-05-27T08:48:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-05-27T08:48:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 871, 2024-05-27T08:48:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 16, 2024-05-27T08:48:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 855, 2024-05-27T08:48:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:31.146 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 848, 2024-05-27T08:48:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-05-27T08:48:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-05-27T08:48:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-05-27T08:48:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 6, 2024-05-27T08:48:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-05-27T08:48:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-05-27T08:48:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-05-27T08:48:31.147 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-05-27T08:48:31.148 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-05-27T08:48:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 0, 2024-05-27T08:48:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 0, 2024-05-27T08:48:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 0, 2024-05-27T08:48:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:31.149 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:31.150 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:48:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T08:48:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-05-27T08:48:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-05-27T08:48:31.151 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-05-27T08:48:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-05-27T08:48:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-05-27T08:48:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-05-27T08:48:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-05-27T08:48:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-05-27T08:48:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-05-27T08:48:31.152 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-05-27T08:48:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-05-27T08:48:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-05-27T08:48:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-05-27T08:48:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-05-27T08:48:31.153 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-05-27T08:48:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-05-27T08:48:31.154 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-05-27T08:48:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:48:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:48:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:31.158 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-05-27T08:48:31.159 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-05-27T08:48:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-05-27T08:48:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 6130, 2024-05-27T08:48:31.160 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-05-27T08:48:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-05-27T08:48:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-05-27T08:48:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-05-27T08:48:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-05-27T08:48:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 9, 2024-05-27T08:48:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-05-27T08:48:31.161 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-05-27T08:48:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-05-27T08:48:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-05-27T08:48:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 13207544, 2024-05-27T08:48:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 13201672, 2024-05-27T08:48:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-05-27T08:48:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.162 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 6130 2024-05-27T08:48:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-05-27T08:48:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 125, 2024-05-27T08:48:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 896, 2024-05-27T08:48:31.163 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 771, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 24, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 24, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 0, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 888, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 888, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 0, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-05-27T08:48:31.164 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-05-27T08:48:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 51772, 2024-05-27T08:48:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215416 2024-05-27T08:48:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-05-27T08:48:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-05-27T08:48:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-05-27T08:48:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-05-27T08:48:31.165 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-05-27T08:48:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-05-27T08:48:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-05-27T08:48:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-05-27T08:48:31.166 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.167 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-05-27T08:48:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-05-27T08:48:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.168 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-05-27T08:48:31.169 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-05-27T08:48:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.170 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:31.171 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.172 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.173 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-05-27T08:48:31.173 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.173 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.173 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.173 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.173 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-05-27T08:48:31.174 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.174 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.174 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.174 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.175 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-05-27T08:48:31.175 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.175 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.175 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.175 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.175 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-05-27T08:48:31.175 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.176 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.176 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.176 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.176 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-05-27T08:48:31.176 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.176 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.176 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.176 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.177 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-05-27T08:48:31.177 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.177 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.177 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.177 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.177 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-05-27T08:48:31.177 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.177 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.178 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.178 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:31.178 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.178 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.178 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.178 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.178 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:31.179 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.179 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.179 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.179 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.179 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-05-27T08:48:31.179 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.179 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.180 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.180 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.180 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:31.180 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.180 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.180 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.180 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.180 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:31.181 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.181 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.181 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.181 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.181 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-05-27T08:48:31.181 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.181 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.181 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.182 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.182 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:31.182 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.182 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.182 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.182 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.182 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:31.182 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.183 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.183 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.183 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.183 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-05-27T08:48:31.183 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.183 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.183 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.184 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.184 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-05-27T08:48:31.184 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.184 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.184 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.184 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.184 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-05-27T08:48:31.184 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.185 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:31.186 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:31.186 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:31.186 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.186 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.186 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.186 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.186 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.187 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-05-27T08:48:31.187 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 1, 2024-05-27T08:48:31.187 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 2, 2024-05-27T08:48:31.187 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 1, 2024-05-27T08:48:31.187 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-05-27T08:48:31.187 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-05-27T08:48:31.187 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-05-27T08:48:31.187 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-05-27T08:48:31.188 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 2949, 2024-05-27T08:48:31.188 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:31.188 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.188 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-05-27T08:48:31.188 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:31.188 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:31.188 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:31.188 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:31.189 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:31.189 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:31.189 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:31.189 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:31.189 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:31.189 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:31.189 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:31.189 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:31.190 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:31.190 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:31.190 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:31.190 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:31.190 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:31.190 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:31.190 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:31.191 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:31.191 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:31.191 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:31.191 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:31.191 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:31.191 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:31.191 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:31.191 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:31.192 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:31.192 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:31.192 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:31.192 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:31.192 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-05-27T08:48:31.192 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:31.192 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:31.192 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:31.193 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:31.193 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 202252, 2024-05-27T08:48:31.193 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 165, 2024-05-27T08:48:31.193 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:31.193 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:31.193 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-05-27T08:48:31.193 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-05-27T08:48:31.193 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:31.194 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:31.194 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:31.194 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:31.194 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:31.194 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 169, 2024-05-27T08:48:31.194 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:31.194 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:31.194 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:31.195 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-05-27T08:48:31.195 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 954639, 2024-05-27T08:48:31.195 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 9622, 2024-05-27T08:48:31.195 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:31.195 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-05-27T08:48:31.195 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:31.195 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-05-27T08:48:31.195 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.196 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-05-27T08:48:31.196 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-05-27T08:48:31.196 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-05-27T08:48:31.196 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 2420, 2024-05-27T08:48:31.196 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 21, 2024-05-27T08:48:31.196 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 3, 2024-05-27T08:48:31.196 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 2420, 2024-05-27T08:48:31.196 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2420, 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.261974398, 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000521477 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2420, 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2421 2024-05-27T08:48:31.197 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.198 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 2420, 2024-05-27T08:48:31.198 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 2420, 2024-05-27T08:48:31.198 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-05-27T08:48:31.198 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-05-27T08:48:31.198 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-05-27T08:48:31.198 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1403, 2024-05-27T08:48:31.198 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-05-27T08:48:31.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 1015, 2024-05-27T08:48:31.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-05-27T08:48:31.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-05-27T08:48:31.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-05-27T08:48:31.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-05-27T08:48:31.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-05-27T08:48:31.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-05-27T08:48:31.199 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 1, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:31.200 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-05-27T08:48:31.201 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-05-27T08:48:31.201 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-05-27T08:48:31.201 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:31.201 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-05-27T08:48:31.201 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:31.201 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-05-27T08:48:31.201 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-05-27T08:48:31.201 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-05-27T08:48:31.202 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-05-27T08:48:31.202 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-05-27T08:48:31.202 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-05-27T08:48:31.202 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-05-27T08:48:31.202 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-05-27T08:48:31.202 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-05-27T08:48:31.202 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-05-27T08:48:31.203 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-05-27T08:48:31.203 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-05-27T08:48:31.203 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-05-27T08:48:31.203 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-05-27T08:48:31.203 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-05-27T08:48:31.203 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-05-27T08:48:31.203 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-05-27T08:48:31.203 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 94, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 11, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 6, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 6, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-05-27T08:48:31.204 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-05-27T08:48:31.205 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.205 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-05-27T08:48:31.205 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-05-27T08:48:31.205 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:31.205 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-05-27T08:48:31.205 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-05-27T08:48:31.205 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.205 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-05-27T08:48:31.206 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-05-27T08:48:31.206 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.206 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-05-27T08:48:31.206 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:31.206 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T08:48:31.206 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-05-27T08:48:31.206 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-05-27T08:48:31.206 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-05-27T08:48:31.207 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-05-27T08:48:31.207 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:31.207 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.207 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:31.207 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:31.207 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-27T08:48:31.207 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:31.207 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 2685, 2024-05-27T08:48:31.208 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 11264271, 2024-05-27T08:48:31.208 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:31.208 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 2685, 2024-05-27T08:48:31.208 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-27T08:48:31.208 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-27T08:48:31.208 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2685, 2024-05-27T08:48:31.208 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 11264271, 2024-05-27T08:48:31.209 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:31.209 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.209 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.209 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.209 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.209 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.209 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:31.209 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:31.210 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-27T08:48:31.210 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:31.210 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-27T08:48:31.210 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-27T08:48:31.210 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:31.210 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:31.210 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2420, 2024-05-27T08:48:31.210 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 9014940, 2024-05-27T08:48:31.211 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2420, 2024-05-27T08:48:31.211 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 9014940, 2024-05-27T08:48:31.211 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:31.211 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.211 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.211 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.211 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.211 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.212 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:31.212 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:31.212 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-05-27T08:48:31.212 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:31.212 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-27T08:48:31.212 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-27T08:48:31.212 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:31.212 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:31.213 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 2420, 2024-05-27T08:48:31.213 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 2420, 2024-05-27T08:48:31.213 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 2420, 2024-05-27T08:48:31.213 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2420, 2024-05-27T08:48:31.213 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:31.213 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.213 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.213 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.214 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.214 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.214 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:31.214 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:31.214 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-27T08:48:31.214 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:31.214 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-27T08:48:31.214 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-27T08:48:31.215 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:31.215 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:31.215 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-27T08:48:31.215 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-27T08:48:31.215 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-05-27T08:48:31.216 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-05-27T08:48:31.216 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:31.216 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.216 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.216 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.216 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.216 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:31.217 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x55aea7c746e0": { 2024-05-27T08:48:31.217 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:31.217 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-05-27T08:48:31.217 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:31.217 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-05-27T08:48:31.217 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-05-27T08:48:31.217 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:31.217 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:31.218 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-27T08:48:31.218 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-27T08:48:31.218 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-05-27T08:48:31.218 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-05-27T08:48:31.218 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:31.218 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:31.218 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:31.218 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:31.219 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.219 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:31.219 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:48:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:31 smithi092 ceph-mon[36049]: pgmap v318: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 542 KiB/s wr, 37 op/s 2024-05-27T08:48:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:31 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:31 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:31.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:31 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:31.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:31 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.050 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-27T08:48:32.050 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-27T08:48:32.050 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-05-27T08:48:32.051 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-05-27T08:48:32.051 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-05-27T08:48:32.051 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:32.051 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:32.051 DEBUG:tasks.check_counter:Getting stats from j 2024-05-27T08:48:32.051 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-27T08:48:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:31 smithi179 ceph-mon[36481]: pgmap v318: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 542 KiB/s wr, 37 op/s 2024-05-27T08:48:32.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:31 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:31 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:31 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:31 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:31 smithi077 ceph-mon[32007]: pgmap v318: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 542 KiB/s wr, 37 op/s 2024-05-27T08:48:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:31 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:31 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:31 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:31 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:32.882 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:48:33.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: pgmap v319: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 815 KiB/s wr, 54 op/s 2024-05-27T08:48:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-05-27T08:48:33.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:32 smithi092 ceph-mon[36049]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-27T08:48:33.258 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: pgmap v319: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 815 KiB/s wr, 54 op/s 2024-05-27T08:48:33.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.259 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.260 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-05-27T08:48:33.260 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:32 smithi077 ceph-mon[32007]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-27T08:48:33.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: pgmap v319: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 815 KiB/s wr, 54 op/s 2024-05-27T08:48:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:33.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-05-27T08:48:33.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:32 smithi179 ceph-mon[36481]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-05-27T08:48:34.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:33 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:33 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:33 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:33 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:33 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-27T08:48:34.085 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:33 smithi077 ceph-mon[32007]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-27T08:48:34.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:33 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:33 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:33 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:33 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:33 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-27T08:48:34.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:33 smithi092 ceph-mon[36049]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-27T08:48:34.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:33 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:33 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:33 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:33 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:33 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi092 to 176.2M 2024-05-27T08:48:34.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:33 smithi179 ceph-mon[36481]: Unable to set osd_memory_target on smithi092 to 184783155: error parsing value: Value '184783155' is below minimum 939524096 2024-05-27T08:48:34.732 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-05-27T08:48:34.732 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:34.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-05-27T08:48:34.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-05-27T08:48:34.732 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-05-27T08:48:34.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-05-27T08:48:34.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-05-27T08:48:34.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-05-27T08:48:34.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.001041508, 2024-05-27T08:48:34.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-05-27T08:48:34.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.000188164, 2024-05-27T08:48:34.733 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:48:34.734 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:34.734 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:34.734 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:34.734 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:34.734 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.734 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:34.734 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:34.734 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:34.735 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:34.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.735 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:48:34.735 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:48:34.735 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.735 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:34.735 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 2, 2024-05-27T08:48:34.735 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 174, 2024-05-27T08:48:34.736 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 320, 2024-05-27T08:48:34.736 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 188016, 2024-05-27T08:48:34.736 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:34.736 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-05-27T08:48:34.736 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.014793036, 2024-05-27T08:48:34.736 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.010582838, 2024-05-27T08:48:34.736 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.001710221, 2024-05-27T08:48:34.736 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-27T08:48:34.737 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:34.737 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 171, 2024-05-27T08:48:34.737 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.010453170, 2024-05-27T08:48:34.737 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000061129 2024-05-27T08:48:34.737 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.737 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:34.737 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:34.737 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:34.738 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:34.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.738 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T08:48:34.738 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 188016 2024-05-27T08:48:34.738 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.738 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:34.738 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 243, 2024-05-27T08:48:34.738 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 230, 2024-05-27T08:48:34.739 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1718464, 2024-05-27T08:48:34.739 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 140672, 2024-05-27T08:48:34.739 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-05-27T08:48:34.739 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-05-27T08:48:34.739 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.045419345, 2024-05-27T08:48:34.739 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.014690374, 2024-05-27T08:48:34.739 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.037478085, 2024-05-27T08:48:34.739 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.004368635, 2024-05-27T08:48:34.740 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:34.740 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 230, 2024-05-27T08:48:34.740 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.010992664, 2024-05-27T08:48:34.740 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000047794 2024-05-27T08:48:34.740 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.740 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:34.740 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:34.740 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:34.741 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:34.741 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.741 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1718464, 2024-05-27T08:48:34.741 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 140672 2024-05-27T08:48:34.741 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.741 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-05-27T08:48:34.741 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-05-27T08:48:34.741 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-05-27T08:48:34.742 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.742 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-05-27T08:48:34.742 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:34.742 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:34.742 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:34.742 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:34.742 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:34.742 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:34.743 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:34.744 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:34.744 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:34.744 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:34.744 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:34.744 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:34.744 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:34.744 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:34.744 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:34.745 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:34.745 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:34.745 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:34.745 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:34.745 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:34.745 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:34.745 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:34.745 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:34.746 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-05-27T08:48:34.746 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:34.746 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:34.746 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:34.746 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:34.746 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148754, 2024-05-27T08:48:34.746 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 31, 2024-05-27T08:48:34.747 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:34.747 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:34.747 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-05-27T08:48:34.747 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-05-27T08:48:34.747 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:34.747 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:34.747 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:34.747 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:34.748 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-05-27T08:48:34.748 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-05-27T08:48:34.748 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:34.748 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:34.748 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:34.748 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-05-27T08:48:34.748 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-05-27T08:48:34.748 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-05-27T08:48:34.749 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:34.749 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-05-27T08:48:34.749 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:34.749 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-05-27T08:48:34.749 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-05-27T08:48:34.749 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:34.749 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-05-27T08:48:34.749 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-05-27T08:48:34.750 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-05-27T08:48:34.750 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 241, 2024-05-27T08:48:34.750 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1681743, 2024-05-27T08:48:34.750 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:34.750 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 241, 2024-05-27T08:48:34.750 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-05-27T08:48:34.750 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-05-27T08:48:34.750 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 241, 2024-05-27T08:48:34.751 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1681743, 2024-05-27T08:48:34.751 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-05-27T08:48:34.751 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-05-27T08:48:34.751 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-05-27T08:48:34.751 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-05-27T08:48:34.751 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:34.751 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-05-27T08:48:34.751 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-05-27T08:48:35.069 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:34 smithi077 ceph-mon[32007]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:48:35.070 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:34 smithi077 ceph-mon[32007]: pgmap v320: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 548 KiB/s wr, 35 op/s 2024-05-27T08:48:35.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:34 smithi092 ceph-mon[36049]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:48:35.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:34 smithi092 ceph-mon[36049]: pgmap v320: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 548 KiB/s wr, 35 op/s 2024-05-27T08:48:35.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:34 smithi179 ceph-mon[36481]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-27T08:48:35.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:34 smithi179 ceph-mon[36481]: pgmap v320: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 548 KiB/s wr, 35 op/s 2024-05-27T08:48:35.489 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-27T08:48:35.489 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-27T08:48:35.489 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-27T08:48:35.490 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-27T08:48:35.490 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-27T08:48:35.490 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-27T08:48:35.490 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-27T08:48:35.490 DEBUG:tasks.check_counter:Getting stats from b 2024-05-27T08:48:35.490 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-27T08:48:35.739 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:48:36.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:36.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:36.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:48:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:48:36.986 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:36 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:37.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:48:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:48:37.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:36 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:37.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-27T08:48:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-27T08:48:37.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-27T08:48:37.130 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:36 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' 2024-05-27T08:48:37.479 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-27T08:48:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 102252, 2024-05-27T08:48:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 146828, 2024-05-27T08:48:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 23072509, 2024-05-27T08:48:37.479 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 102048223, 2024-05-27T08:48:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 11, 2024-05-27T08:48:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-27T08:48:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 5.748480457, 2024-05-27T08:48:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 2.597304654, 2024-05-27T08:48:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 3.503698854, 2024-05-27T08:48:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.201309736, 2024-05-27T08:48:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:37.480 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 146830, 2024-05-27T08:48:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.099653260, 2024-05-27T08:48:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000014299 2024-05-27T08:48:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 232514, 2024-05-27T08:48:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.032497112, 2024-05-27T08:48:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000139 2024-05-27T08:48:37.481 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-27T08:48:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 314880 2024-05-27T08:48:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 110069, 2024-05-27T08:48:37.482 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 148500, 2024-05-27T08:48:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 33316019, 2024-05-27T08:48:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 142495745, 2024-05-27T08:48:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 14, 2024-05-27T08:48:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 7, 2024-05-27T08:48:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 6.308697529, 2024-05-27T08:48:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 2.821331539, 2024-05-27T08:48:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 3.952491852, 2024-05-27T08:48:37.483 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.272307602, 2024-05-27T08:48:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 148498, 2024-05-27T08:48:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.353097953, 2024-05-27T08:48:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000015845 2024-05-27T08:48:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 224507, 2024-05-27T08:48:37.484 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.032789769, 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000146 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 22806784, 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 64364208 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 58733, 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 76430, 2024-05-27T08:48:37.485 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 15152923, 2024-05-27T08:48:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 49573218, 2024-05-27T08:48:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 46, 2024-05-27T08:48:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-27T08:48:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 3.376062330, 2024-05-27T08:48:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 1.399832727, 2024-05-27T08:48:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 2.128999325, 2024-05-27T08:48:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.139803940, 2024-05-27T08:48:37.486 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 76425, 2024-05-27T08:48:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.333148014, 2024-05-27T08:48:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000017443 2024-05-27T08:48:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 125362, 2024-05-27T08:48:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.019819014, 2024-05-27T08:48:37.487 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000158 2024-05-27T08:48:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2007888, 2024-05-27T08:48:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 179856 2024-05-27T08:48:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-27T08:48:37.488 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-27T08:48:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-27T08:48:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-27T08:48:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-27T08:48:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-27T08:48:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 34515, 2024-05-27T08:48:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 17.722919737, 2024-05-27T08:48:37.489 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000513484 2024-05-27T08:48:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-27T08:48:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-27T08:48:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 5285, 2024-05-27T08:48:37.490 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.379867329, 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000071876 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 41950, 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 37857, 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 37857, 2024-05-27T08:48:37.491 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 226.730940893, 2024-05-27T08:48:37.492 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.005989141 2024-05-27T08:48:37.492 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.492 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-27T08:48:37.492 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 4093, 2024-05-27T08:48:37.492 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:37.492 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:37.492 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 30, 2024-05-27T08:48:37.492 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 2, 2024-05-27T08:48:37.493 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 1, 2024-05-27T08:48:37.493 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 209, 2024-05-27T08:48:37.493 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 92, 2024-05-27T08:48:37.493 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 83, 2024-05-27T08:48:37.493 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 34, 2024-05-27T08:48:37.493 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 77, 2024-05-27T08:48:37.493 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 481, 2024-05-27T08:48:37.493 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 19, 2024-05-27T08:48:37.494 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 19, 2024-05-27T08:48:37.494 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 20, 2024-05-27T08:48:37.494 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 65801, 2024-05-27T08:48:37.494 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 47402, 2024-05-27T08:48:37.494 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 4088, 2024-05-27T08:48:37.494 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 67, 2024-05-27T08:48:37.494 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 195, 2024-05-27T08:48:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 15292, 2024-05-27T08:48:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 10, 2024-05-27T08:48:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 14, 2024-05-27T08:48:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 246, 2024-05-27T08:48:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-05-27T08:48:37.495 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-05-27T08:48:37.496 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:37.496 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:37.496 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:37.496 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 67, 2024-05-27T08:48:37.496 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 21137952, 2024-05-27T08:48:37.496 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 6, 2024-05-27T08:48:37.496 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:37.496 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:37.497 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:37.497 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:37.497 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:37.497 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:37.497 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:37.497 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:37.497 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 10, 2024-05-27T08:48:37.497 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 12409, 2024-05-27T08:48:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 10471, 2024-05-27T08:48:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 17773, 2024-05-27T08:48:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 2889, 2024-05-27T08:48:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 1747, 2024-05-27T08:48:37.498 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 19454 2024-05-27T08:48:37.499 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.499 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-27T08:48:37.499 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 55, 2024-05-27T08:48:37.499 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-05-27T08:48:37.499 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-05-27T08:48:37.499 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 67, 2024-05-27T08:48:37.499 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 1732, 2024-05-27T08:48:37.500 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 1, 2024-05-27T08:48:37.500 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:37.500 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:37.500 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 2792, 2024-05-27T08:48:37.500 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 2791, 2024-05-27T08:48:37.500 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:37.500 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-27T08:48:37.500 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:37.501 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:37.501 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:37.501 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-27T08:48:37.501 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-27T08:48:37.501 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 7, 2024-05-27T08:48:37.501 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 387, 2024-05-27T08:48:37.501 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:37.501 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 16, 2024-05-27T08:48:37.502 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-27T08:48:37.502 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 3, 2024-05-27T08:48:37.502 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:37.502 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-27T08:48:37.502 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.502 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-27T08:48:37.503 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-27T08:48:37.503 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 26931, 2024-05-27T08:48:37.503 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-05-27T08:48:37.503 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-05-27T08:48:37.503 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 26931, 2024-05-27T08:48:37.504 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-27T08:48:37.504 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-05-27T08:48:37.504 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 44, 2024-05-27T08:48:37.504 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-05-27T08:48:37.504 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-05-27T08:48:37.504 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 44, 2024-05-27T08:48:37.505 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 4, 2024-05-27T08:48:37.505 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-27T08:48:37.505 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-05-27T08:48:37.505 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-05-27T08:48:37.505 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 140147670, 2024-05-27T08:48:37.505 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-05-27T08:48:37.505 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-27T08:48:37.505 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 17369, 2024-05-27T08:48:37.506 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 28.193196559, 2024-05-27T08:48:37.506 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001623190 2024-05-27T08:48:37.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.506 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-05-27T08:48:37.506 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.506 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-27T08:48:37.506 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 115, 2024-05-27T08:48:37.506 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 3184, 2024-05-27T08:48:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 3069, 2024-05-27T08:48:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 54, 2024-05-27T08:48:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 66, 2024-05-27T08:48:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 12, 2024-05-27T08:48:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 205, 2024-05-27T08:48:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 7227, 2024-05-27T08:48:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 7022, 2024-05-27T08:48:37.507 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 18, 2024-05-27T08:48:37.508 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 3337, 2024-05-27T08:48:37.508 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 3319, 2024-05-27T08:48:37.508 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 73552, 2024-05-27T08:48:37.508 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 231800 2024-05-27T08:48:37.508 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.508 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-27T08:48:37.508 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 62336, 2024-05-27T08:48:37.509 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 19588, 2024-05-27T08:48:37.509 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 41950, 2024-05-27T08:48:37.509 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 485, 2024-05-27T08:48:37.509 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 39228, 2024-05-27T08:48:37.509 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-27T08:48:37.509 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2851, 2024-05-27T08:48:37.509 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 4.120846310, 2024-05-27T08:48:37.509 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001445403 2024-05-27T08:48:37.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.510 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-27T08:48:37.510 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 12307, 2024-05-27T08:48:37.510 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 57.789083152, 2024-05-27T08:48:37.510 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.004695627 2024-05-27T08:48:37.510 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.510 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:37.510 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.511 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.511 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.511 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.511 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-27T08:48:37.511 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.512 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.512 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.512 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.512 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-27T08:48:37.512 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 6998, 2024-05-27T08:48:37.512 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 80.282369410, 2024-05-27T08:48:37.513 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.011472187 2024-05-27T08:48:37.513 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.513 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:37.513 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.514 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.514 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.514 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.514 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-27T08:48:37.514 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.515 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.515 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.515 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-27T08:48:37.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.515 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.515 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.516 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.517 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.517 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.518 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-27T08:48:37.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 10, 2024-05-27T08:48:37.518 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.001661243, 2024-05-27T08:48:37.518 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000166124 2024-05-27T08:48:37.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.519 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-27T08:48:37.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 20, 2024-05-27T08:48:37.519 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.027656233, 2024-05-27T08:48:37.519 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001382811 2024-05-27T08:48:37.519 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.520 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-27T08:48:37.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.520 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.520 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.520 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-27T08:48:37.520 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 10, 2024-05-27T08:48:37.520 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3.510522139, 2024-05-27T08:48:37.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.351052213 2024-05-27T08:48:37.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.521 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-27T08:48:37.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-05-27T08:48:37.521 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000223601, 2024-05-27T08:48:37.521 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000223601 2024-05-27T08:48:37.521 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.521 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-27T08:48:37.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 12330, 2024-05-27T08:48:37.522 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 13.525093858, 2024-05-27T08:48:37.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001096925 2024-05-27T08:48:37.522 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.522 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-27T08:48:37.522 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 466, 2024-05-27T08:48:37.523 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 63.259429572, 2024-05-27T08:48:37.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.135749848 2024-05-27T08:48:37.523 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.523 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:37.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.523 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.523 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-27T08:48:37.524 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.525 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.525 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.525 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.525 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:37.525 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-05-27T08:48:37.526 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.043716255, 2024-05-27T08:48:37.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.010929063 2024-05-27T08:48:37.526 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.526 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:37.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.526 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.526 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.527 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-27T08:48:37.527 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1, 2024-05-27T08:48:37.527 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000859053, 2024-05-27T08:48:37.527 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000859053 2024-05-27T08:48:37.527 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.527 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:37.527 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 60, 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.025639337, 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000427322 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.528 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-27T08:48:37.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.529 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.529 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.529 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-27T08:48:37.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 7, 2024-05-27T08:48:37.529 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.021680319, 2024-05-27T08:48:37.529 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.003097188 2024-05-27T08:48:37.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.530 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-27T08:48:37.530 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.530 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.530 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.530 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.530 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-27T08:48:37.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2792, 2024-05-27T08:48:37.531 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 4.122160411, 2024-05-27T08:48:37.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001476418 2024-05-27T08:48:37.531 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.531 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:37.531 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:37.531 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:37.531 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 5, 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 3, 2024-05-27T08:48:37.532 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-05-27T08:48:37.533 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-27T08:48:37.533 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 4786, 2024-05-27T08:48:37.533 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 2393, 2024-05-27T08:48:37.533 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 596, 2024-05-27T08:48:37.533 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:37.534 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.534 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-27T08:48:37.534 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:37.534 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:37.534 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:37.534 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:37.535 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:37.535 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:37.535 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:37.535 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:37.535 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:37.535 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:37.535 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:37.536 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:37.536 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:37.536 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:37.536 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:37.536 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:37.537 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:37.537 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:37.537 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:37.537 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:37.537 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:37.537 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:37.537 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:37.537 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:37.538 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:37.538 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:37.538 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:37.538 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:37.538 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:37.538 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:37.539 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:37.539 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-27T08:48:37.539 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:37.539 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:37.539 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:37.539 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:37.539 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 349555, 2024-05-27T08:48:37.540 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 169, 2024-05-27T08:48:37.540 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:37.540 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:37.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-27T08:48:37.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-27T08:48:37.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:37.540 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:37.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:37.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:37.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:37.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 169, 2024-05-27T08:48:37.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:37.541 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:37.541 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:37.541 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-27T08:48:37.542 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 742663, 2024-05-27T08:48:37.542 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 7901, 2024-05-27T08:48:37.542 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:37.542 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-27T08:48:37.542 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:37.542 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-27T08:48:37.542 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.542 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-27T08:48:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-27T08:48:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-27T08:48:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 21428, 2024-05-27T08:48:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 136697417, 2024-05-27T08:48:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 37, 2024-05-27T08:48:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 21428, 2024-05-27T08:48:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-27T08:48:37.543 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 21428, 2024-05-27T08:48:37.544 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 51.805502679, 2024-05-27T08:48:37.544 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.002417654 2024-05-27T08:48:37.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.544 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-27T08:48:37.544 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-27T08:48:37.544 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 21428, 2024-05-27T08:48:37.544 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 21774 2024-05-27T08:48:37.544 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 21428, 2024-05-27T08:48:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 148, 2024-05-27T08:48:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 21280, 2024-05-27T08:48:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-27T08:48:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-27T08:48:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-05-27T08:48:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 11, 2024-05-27T08:48:37.545 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 148, 2024-05-27T08:48:37.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 17517, 2024-05-27T08:48:37.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 126, 2024-05-27T08:48:37.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-27T08:48:37.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-27T08:48:37.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-27T08:48:37.546 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 2, 2024-05-27T08:48:37.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 3343, 2024-05-27T08:48:37.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-27T08:48:37.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:37.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:37.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-05-27T08:48:37.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 22, 2024-05-27T08:48:37.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:37.547 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:37.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:37.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-27T08:48:37.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-27T08:48:37.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-27T08:48:37.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:37.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-27T08:48:37.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:37.548 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-27T08:48:37.549 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-27T08:48:37.549 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-27T08:48:37.549 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-27T08:48:37.549 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-27T08:48:37.549 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-27T08:48:37.549 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-27T08:48:37.549 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-27T08:48:37.549 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-27T08:48:37.550 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-27T08:48:37.550 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-27T08:48:37.550 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-27T08:48:37.550 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-27T08:48:37.550 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-27T08:48:37.550 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-27T08:48:37.550 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-27T08:48:37.550 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-27T08:48:37.551 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 94, 2024-05-27T08:48:37.551 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-27T08:48:37.551 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 11, 2024-05-27T08:48:37.551 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-05-27T08:48:37.551 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-05-27T08:48:37.551 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-27T08:48:37.551 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-27T08:48:37.551 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 499, 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 106 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-27T08:48:37.552 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.553 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-27T08:48:37.553 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 394 2024-05-27T08:48:37.553 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.553 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-27T08:48:37.553 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:37.553 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 6, 2024-05-27T08:48:37.553 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-27T08:48:37.553 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 6, 2024-05-27T08:48:37.554 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 2791, 2024-05-27T08:48:37.554 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 2791, 2024-05-27T08:48:37.554 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:37.554 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.554 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:37.554 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:37.554 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-27T08:48:37.554 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:37.555 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 271050, 2024-05-27T08:48:37.555 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 44165767, 2024-05-27T08:48:37.555 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:37.555 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 271050, 2024-05-27T08:48:37.555 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:37.555 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:37.555 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 271050, 2024-05-27T08:48:37.555 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 44165767, 2024-05-27T08:48:37.556 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:37.556 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.556 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.556 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.556 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.556 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.556 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:37.556 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 21428, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 136979444, 2024-05-27T08:48:37.557 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 21428, 2024-05-27T08:48:37.558 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 136979444, 2024-05-27T08:48:37.558 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:37.558 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.558 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.558 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.558 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.558 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:37.559 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 21428, 2024-05-27T08:48:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 21428, 2024-05-27T08:48:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 21428, 2024-05-27T08:48:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 21428, 2024-05-27T08:48:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.560 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.561 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.561 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:37.561 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:37.561 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-27T08:48:37.561 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:37.561 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 2791, 2024-05-27T08:48:37.561 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 282027, 2024-05-27T08:48:37.561 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 2791, 2024-05-27T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 148, 2024-05-27T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 282027, 2024-05-27T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.562 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.563 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.563 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.563 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:37.563 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55637d4a2500": { 2024-05-27T08:48:37.563 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 73610, 2024-05-27T08:48:37.563 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-27T08:48:37.563 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:37.563 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 26931, 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 135953366, 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 26931, 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 17369, 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 135879756, 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:37.564 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:37.565 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:37.565 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:37.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.565 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:37.565 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-27T08:48:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:37 smithi092 ceph-mon[36049]: Adjusting osd_memory_target on smithi077 to 176.1M 2024-05-27T08:48:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:37 smithi092 ceph-mon[36049]: Unable to set osd_memory_target on smithi077 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-27T08:48:37.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:37 smithi092 ceph-mon[36049]: pgmap v321: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 548 KiB/s wr, 35 op/s 2024-05-27T08:48:38.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:37 smithi179 ceph-mon[36481]: Adjusting osd_memory_target on smithi077 to 176.1M 2024-05-27T08:48:38.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:37 smithi179 ceph-mon[36481]: Unable to set osd_memory_target on smithi077 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-27T08:48:38.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:37 smithi179 ceph-mon[36481]: pgmap v321: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 548 KiB/s wr, 35 op/s 2024-05-27T08:48:38.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:37 smithi077 ceph-mon[32007]: Adjusting osd_memory_target on smithi077 to 176.1M 2024-05-27T08:48:38.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:37 smithi077 ceph-mon[32007]: Unable to set osd_memory_target on smithi077 to 184753049: error parsing value: Value '184753049' is below minimum 939524096 2024-05-27T08:48:38.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:37 smithi077 ceph-mon[32007]: pgmap v321: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 548 KiB/s wr, 35 op/s 2024-05-27T08:48:38.308 INFO:tasks.check_counter:Daemon mds.b mds.exported=14 2024-05-27T08:48:38.309 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-27T08:48:38.309 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=55 2024-05-27T08:48:38.309 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-27T08:48:38.309 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-27T08:48:38.309 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-05-27T08:48:38.309 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-05-27T08:48:38.309 DEBUG:tasks.check_counter:Getting stats from e 2024-05-27T08:48:38.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-27T08:48:38.555 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:48:39.904 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:39 smithi092 ceph-mon[36049]: pgmap v322: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 805 KiB/s wr, 51 op/s 2024-05-27T08:48:40.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:39 smithi179 ceph-mon[36481]: pgmap v322: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 805 KiB/s wr, 51 op/s 2024-05-27T08:48:40.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:39 smithi077 ceph-mon[32007]: pgmap v322: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 805 KiB/s wr, 51 op/s 2024-05-27T08:48:40.223 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-27T08:48:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1792, 2024-05-27T08:48:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1794, 2024-05-27T08:48:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 63437613, 2024-05-27T08:48:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 569866, 2024-05-27T08:48:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 8, 2024-05-27T08:48:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-05-27T08:48:40.224 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.171587650, 2024-05-27T08:48:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.052846187, 2024-05-27T08:48:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.062531885, 2024-05-27T08:48:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.043028216, 2024-05-27T08:48:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1792, 2024-05-27T08:48:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.058868275, 2024-05-27T08:48:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000032850 2024-05-27T08:48:40.225 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:48:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:48:40.226 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 440, 2024-05-27T08:48:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 612, 2024-05-27T08:48:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 42055604, 2024-05-27T08:48:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 601254, 2024-05-27T08:48:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 8, 2024-05-27T08:48:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-05-27T08:48:40.227 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.064321169, 2024-05-27T08:48:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.014660553, 2024-05-27T08:48:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.035023801, 2024-05-27T08:48:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.010297103, 2024-05-27T08:48:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:40.228 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 609, 2024-05-27T08:48:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.023914402, 2024-05-27T08:48:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000039268 2024-05-27T08:48:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.229 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T08:48:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 461904 2024-05-27T08:48:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 837, 2024-05-27T08:48:40.230 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 810, 2024-05-27T08:48:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 33106568, 2024-05-27T08:48:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 322313, 2024-05-27T08:48:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 4, 2024-05-27T08:48:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.108152955, 2024-05-27T08:48:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.026070338, 2024-05-27T08:48:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.068792512, 2024-05-27T08:48:40.231 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.020855474, 2024-05-27T08:48:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 810, 2024-05-27T08:48:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.028133972, 2024-05-27T08:48:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000034733 2024-05-27T08:48:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.232 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1835232, 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 142960 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-27T08:48:40.233 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-27T08:48:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-27T08:48:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-27T08:48:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4705, 2024-05-27T08:48:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.101457333, 2024-05-27T08:48:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000021563 2024-05-27T08:48:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.234 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-27T08:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-27T08:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.235 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-27T08:48:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-05-27T08:48:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-05-27T08:48:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-27T08:48:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.236 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-27T08:48:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-05-27T08:48:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-05-27T08:48:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-05-27T08:48:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-05-27T08:48:40.237 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 2879, 2024-05-27T08:48:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 0, 2024-05-27T08:48:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 2879, 2024-05-27T08:48:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 2879, 2024-05-27T08:48:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-05-27T08:48:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-05-27T08:48:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-05-27T08:48:40.238 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 16, 2024-05-27T08:48:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-05-27T08:48:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-05-27T08:48:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-05-27T08:48:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-05-27T08:48:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-05-27T08:48:40.239 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-05-27T08:48:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-27T08:48:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-05-27T08:48:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-05-27T08:48:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-05-27T08:48:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-05-27T08:48:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:40.240 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 67, 2024-05-27T08:48:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 21137952, 2024-05-27T08:48:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 6, 2024-05-27T08:48:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:40.241 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:40.242 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:48:40.243 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T08:48:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-05-27T08:48:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-05-27T08:48:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-05-27T08:48:40.244 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-05-27T08:48:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-05-27T08:48:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-27T08:48:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-05-27T08:48:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-05-27T08:48:40.245 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-05-27T08:48:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-05-27T08:48:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-05-27T08:48:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-27T08:48:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-05-27T08:48:40.246 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-05-27T08:48:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-27T08:48:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:40.247 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-27T08:48:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-27T08:48:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:48:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:48:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:40.248 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-27T08:48:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-27T08:48:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-27T08:48:40.249 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-27T08:48:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-05-27T08:48:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-05-27T08:48:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-05-27T08:48:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 26934, 2024-05-27T08:48:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-27T08:48:40.250 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-05-27T08:48:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-05-27T08:48:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-05-27T08:48:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-05-27T08:48:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 44, 2024-05-27T08:48:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 4, 2024-05-27T08:48:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-27T08:48:40.251 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-05-27T08:48:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-05-27T08:48:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 140181945, 2024-05-27T08:48:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 140169523, 2024-05-27T08:48:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-27T08:48:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.252 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 26934 2024-05-27T08:48:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-27T08:48:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 86, 2024-05-27T08:48:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 3020, 2024-05-27T08:48:40.253 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 2934, 2024-05-27T08:48:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 39, 2024-05-27T08:48:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 49, 2024-05-27T08:48:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 10, 2024-05-27T08:48:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 2879, 2024-05-27T08:48:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 3077, 2024-05-27T08:48:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 198, 2024-05-27T08:48:40.254 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 58636, 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215416 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-05-27T08:48:40.255 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-05-27T08:48:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-05-27T08:48:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-05-27T08:48:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-27T08:48:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.256 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-27T08:48:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.257 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.258 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-27T08:48:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-27T08:48:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.259 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-27T08:48:40.260 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-27T08:48:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.261 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:40.262 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-27T08:48:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.263 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-27T08:48:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.264 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-27T08:48:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-27T08:48:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.265 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-27T08:48:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.266 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-27T08:48:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-27T08:48:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.267 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.268 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.269 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.270 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-27T08:48:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.271 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.272 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-27T08:48:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-27T08:48:40.273 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.274 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-27T08:48:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.275 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.276 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-27T08:48:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-05-27T08:48:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 5, 2024-05-27T08:48:40.277 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 3, 2024-05-27T08:48:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 4, 2024-05-27T08:48:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-27T08:48:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-05-27T08:48:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-05-27T08:48:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 3375, 2024-05-27T08:48:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.278 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:40.279 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:40.280 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:40.281 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-27T08:48:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:40.282 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 238964, 2024-05-27T08:48:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 260, 2024-05-27T08:48:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-27T08:48:40.283 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-27T08:48:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 169, 2024-05-27T08:48:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:40.284 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-27T08:48:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 2473671, 2024-05-27T08:48:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 15957, 2024-05-27T08:48:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-27T08:48:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:40.285 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-27T08:48:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-27T08:48:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-27T08:48:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-27T08:48:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 2794, 2024-05-27T08:48:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 26, 2024-05-27T08:48:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 4, 2024-05-27T08:48:40.286 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 2794, 2024-05-27T08:48:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-27T08:48:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2794, 2024-05-27T08:48:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.616953252, 2024-05-27T08:48:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000578723 2024-05-27T08:48:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-27T08:48:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-27T08:48:40.287 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2794, 2024-05-27T08:48:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2795 2024-05-27T08:48:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 2794, 2024-05-27T08:48:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 2794, 2024-05-27T08:48:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-05-27T08:48:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-27T08:48:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-27T08:48:40.288 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 1610, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 1181, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-05-27T08:48:40.289 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-05-27T08:48:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-27T08:48:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 2, 2024-05-27T08:48:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-05-27T08:48:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:40.290 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-27T08:48:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-27T08:48:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-27T08:48:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-27T08:48:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-27T08:48:40.291 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-27T08:48:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-27T08:48:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-27T08:48:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-27T08:48:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-27T08:48:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-27T08:48:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-27T08:48:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-27T08:48:40.292 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-27T08:48:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-27T08:48:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-27T08:48:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-27T08:48:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-27T08:48:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-27T08:48:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-27T08:48:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-27T08:48:40.293 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 94, 2024-05-27T08:48:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-27T08:48:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 11, 2024-05-27T08:48:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 10, 2024-05-27T08:48:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 10, 2024-05-27T08:48:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-27T08:48:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-27T08:48:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-05-27T08:48:40.294 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-05-27T08:48:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-05-27T08:48:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-27T08:48:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-05-27T08:48:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-27T08:48:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-27T08:48:40.295 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-05-27T08:48:40.296 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-05-27T08:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-05-27T08:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-27T08:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:40.297 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 3065, 2024-05-27T08:48:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 138307171, 2024-05-27T08:48:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 3065, 2024-05-27T08:48:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3065, 2024-05-27T08:48:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 138307171, 2024-05-27T08:48:40.298 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:40.299 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-27T08:48:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2794, 2024-05-27T08:48:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 135987651, 2024-05-27T08:48:40.300 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2794, 2024-05-27T08:48:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 135987651, 2024-05-27T08:48:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.301 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-27T08:48:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:40.302 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2794, 2024-05-27T08:48:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 2794, 2024-05-27T08:48:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2794, 2024-05-27T08:48:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 2794, 2024-05-27T08:48:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.303 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-27T08:48:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:40.304 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.305 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x55c91830c6e0": { 2024-05-27T08:48:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-27T08:48:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:40.306 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:40.307 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:40.307 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:40.307 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:40.307 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:40.307 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-27T08:48:40.307 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-27T08:48:40.307 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:40.307 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:40.308 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:40.308 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:40.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.308 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:40.308 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-27T08:48:40.971 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-27T08:48:40.971 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-27T08:48:40.971 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-27T08:48:40.971 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-27T08:48:40.971 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-05-27T08:48:40.971 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:40.972 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:40.972 DEBUG:tasks.check_counter:Getting stats from h 2024-05-27T08:48:40.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-27T08:48:41.210 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:48:41.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:41 smithi092 ceph-mon[36049]: pgmap v323: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 530 KiB/s wr, 33 op/s 2024-05-27T08:48:42.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:41 smithi179 ceph-mon[36481]: pgmap v323: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 530 KiB/s wr, 33 op/s 2024-05-27T08:48:42.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:41 smithi077 ceph-mon[32007]: pgmap v323: 97 pgs: 97 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 530 KiB/s wr, 33 op/s 2024-05-27T08:48:42.801 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-27T08:48:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 67663, 2024-05-27T08:48:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 38956, 2024-05-27T08:48:42.801 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 23175968, 2024-05-27T08:48:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 12915514, 2024-05-27T08:48:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 12, 2024-05-27T08:48:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-05-27T08:48:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 3.147421007, 2024-05-27T08:48:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 1.369230598, 2024-05-27T08:48:42.802 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 2.397202850, 2024-05-27T08:48:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.040911219, 2024-05-27T08:48:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 38956, 2024-05-27T08:48:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.532474232, 2024-05-27T08:48:42.803 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000013668 2024-05-27T08:48:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 102957, 2024-05-27T08:48:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.014064517, 2024-05-27T08:48:42.804 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000136 2024-05-27T08:48:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1847248, 2024-05-27T08:48:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 146960 2024-05-27T08:48:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.805 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 10488, 2024-05-27T08:48:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 14806, 2024-05-27T08:48:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 3825772, 2024-05-27T08:48:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 11138154, 2024-05-27T08:48:42.806 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 18, 2024-05-27T08:48:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-05-27T08:48:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.660148749, 2024-05-27T08:48:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.303247179, 2024-05-27T08:48:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.426166634, 2024-05-27T08:48:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.007619156, 2024-05-27T08:48:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:42.807 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 14803, 2024-05-27T08:48:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.248855509, 2024-05-27T08:48:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000016811 2024-05-27T08:48:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 22756, 2024-05-27T08:48:42.808 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.002953207, 2024-05-27T08:48:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000129 2024-05-27T08:48:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 3491136, 2024-05-27T08:48:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 10746656 2024-05-27T08:48:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:42.809 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1672, 2024-05-27T08:48:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1793, 2024-05-27T08:48:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 506146, 2024-05-27T08:48:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 5902818, 2024-05-27T08:48:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 9, 2024-05-27T08:48:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-05-27T08:48:42.810 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.124254344, 2024-05-27T08:48:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.049334727, 2024-05-27T08:48:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.035968469, 2024-05-27T08:48:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.032869239, 2024-05-27T08:48:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1791, 2024-05-27T08:48:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.071527278, 2024-05-27T08:48:42.811 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000039937 2024-05-27T08:48:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 609, 2024-05-27T08:48:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000090356, 2024-05-27T08:48:42.812 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000148 2024-05-27T08:48:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 160, 2024-05-27T08:48:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 314208 2024-05-27T08:48:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-27T08:48:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-27T08:48:42.813 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-27T08:48:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-27T08:48:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-27T08:48:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-27T08:48:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3110, 2024-05-27T08:48:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.543614380, 2024-05-27T08:48:42.814 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000496339 2024-05-27T08:48:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-27T08:48:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-27T08:48:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2040, 2024-05-27T08:48:42.815 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.088659553, 2024-05-27T08:48:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000043460 2024-05-27T08:48:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-27T08:48:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 6363, 2024-05-27T08:48:42.816 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 6177, 2024-05-27T08:48:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-27T08:48:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 6177, 2024-05-27T08:48:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 58.673333468, 2024-05-27T08:48:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.009498677 2024-05-27T08:48:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-27T08:48:42.817 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 186, 2024-05-27T08:48:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 11, 2024-05-27T08:48:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-05-27T08:48:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-05-27T08:48:42.818 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 146, 2024-05-27T08:48:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 33, 2024-05-27T08:48:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 113, 2024-05-27T08:48:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 98, 2024-05-27T08:48:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 45, 2024-05-27T08:48:42.819 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 6, 2024-05-27T08:48:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 6, 2024-05-27T08:48:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 10, 2024-05-27T08:48:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 11395, 2024-05-27T08:48:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 6021, 2024-05-27T08:48:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 186, 2024-05-27T08:48:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 41, 2024-05-27T08:48:42.820 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-05-27T08:48:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 1006, 2024-05-27T08:48:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-27T08:48:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-05-27T08:48:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-05-27T08:48:42.821 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 5, 2024-05-27T08:48:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 64, 2024-05-27T08:48:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 67, 2024-05-27T08:48:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 21137952, 2024-05-27T08:48:42.822 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 6, 2024-05-27T08:48:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:42.823 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:48:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 721, 2024-05-27T08:48:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 3807, 2024-05-27T08:48:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:42.824 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 2264, 2024-05-27T08:48:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 949, 2024-05-27T08:48:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 559, 2024-05-27T08:48:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 6202 2024-05-27T08:48:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-27T08:48:42.825 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 331, 2024-05-27T08:48:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 767, 2024-05-27T08:48:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 390, 2024-05-27T08:48:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 788, 2024-05-27T08:48:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 362, 2024-05-27T08:48:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-27T08:48:42.826 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 768, 2024-05-27T08:48:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 768, 2024-05-27T08:48:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-27T08:48:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:42.827 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-27T08:48:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-27T08:48:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 7, 2024-05-27T08:48:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 145, 2024-05-27T08:48:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:42.828 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-27T08:48:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-27T08:48:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.829 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-27T08:48:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-27T08:48:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 6155, 2024-05-27T08:48:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-05-27T08:48:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-05-27T08:48:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 6155, 2024-05-27T08:48:42.830 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-27T08:48:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-05-27T08:48:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 9, 2024-05-27T08:48:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-05-27T08:48:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-05-27T08:48:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 9, 2024-05-27T08:48:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 2, 2024-05-27T08:48:42.831 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-27T08:48:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-05-27T08:48:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-05-27T08:48:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 13260279, 2024-05-27T08:48:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 0, 2024-05-27T08:48:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-27T08:48:42.832 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1465, 2024-05-27T08:48:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2.865027492, 2024-05-27T08:48:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001955650 2024-05-27T08:48:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 0 2024-05-27T08:48:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-27T08:48:42.833 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 126, 2024-05-27T08:48:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 917, 2024-05-27T08:48:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 791, 2024-05-27T08:48:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 42, 2024-05-27T08:48:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 421, 2024-05-27T08:48:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 379, 2024-05-27T08:48:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 142, 2024-05-27T08:48:42.834 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 1059, 2024-05-27T08:48:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 917, 2024-05-27T08:48:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 4, 2024-05-27T08:48:42.835 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 842, 2024-05-27T08:48:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 838, 2024-05-27T08:48:42.836 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 58480, 2024-05-27T08:48:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 223608 2024-05-27T08:48:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-27T08:48:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 10219, 2024-05-27T08:48:42.837 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 361, 2024-05-27T08:48:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 6363, 2024-05-27T08:48:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 36, 2024-05-27T08:48:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 1741, 2024-05-27T08:48:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-27T08:48:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 788, 2024-05-27T08:48:42.838 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 55.567584573, 2024-05-27T08:48:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.070517239 2024-05-27T08:48:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-27T08:48:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4, 2024-05-27T08:48:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.005572836, 2024-05-27T08:48:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001393209 2024-05-27T08:48:42.839 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-27T08:48:42.840 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-27T08:48:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 735, 2024-05-27T08:48:42.841 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.187797850, 2024-05-27T08:48:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000255507 2024-05-27T08:48:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.842 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-27T08:48:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2, 2024-05-27T08:48:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000317157, 2024-05-27T08:48:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000158578 2024-05-27T08:48:42.843 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-27T08:48:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:42.844 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.845 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.846 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-27T08:48:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.847 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-27T08:48:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.848 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-27T08:48:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.849 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-27T08:48:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.850 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-27T08:48:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-27T08:48:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3880, 2024-05-27T08:48:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.460966765, 2024-05-27T08:48:42.851 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000376537 2024-05-27T08:48:42.852 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.852 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-27T08:48:42.852 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.852 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.852 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.853 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:42.853 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.853 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.853 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.853 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.853 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:42.854 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.854 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.854 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.854 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.854 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-27T08:48:42.854 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.854 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.854 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.855 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:42.855 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.855 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.855 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.855 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.855 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:42.855 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.856 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.856 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.856 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-27T08:48:42.856 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.856 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.856 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.856 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.857 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:42.857 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.857 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.857 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.857 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.857 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:42.857 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.857 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.858 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.858 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.858 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-27T08:48:42.858 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.859 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.859 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.859 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-27T08:48:42.859 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.859 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.859 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.859 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.860 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-27T08:48:42.860 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.860 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.860 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.860 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.860 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-27T08:48:42.860 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 768, 2024-05-27T08:48:42.860 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.451094287, 2024-05-27T08:48:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001889445 2024-05-27T08:48:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.861 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.862 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.862 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.862 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-27T08:48:42.862 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 1, 2024-05-27T08:48:42.862 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 2, 2024-05-27T08:48:42.862 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 1, 2024-05-27T08:48:42.863 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 1, 2024-05-27T08:48:42.863 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-27T08:48:42.863 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 2023, 2024-05-27T08:48:42.863 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 2023, 2024-05-27T08:48:42.863 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 627, 2024-05-27T08:48:42.863 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:42.863 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.863 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-27T08:48:42.864 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:42.864 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:42.864 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:42.864 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:42.864 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:42.864 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:42.864 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:42.865 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:42.865 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:42.865 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:42.865 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:42.865 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:42.865 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:42.865 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:42.866 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:42.866 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:42.866 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:42.866 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:42.866 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:42.866 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:42.866 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:42.867 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:42.867 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:42.867 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:42.867 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:42.867 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:42.867 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:42.867 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:42.868 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:42.868 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:42.868 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:42.868 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-27T08:48:42.868 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:42.868 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:42.868 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:42.869 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:42.869 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 359984, 2024-05-27T08:48:42.869 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 206, 2024-05-27T08:48:42.869 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 88, 2024-05-27T08:48:42.869 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 1, 2024-05-27T08:48:42.869 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-27T08:48:42.869 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-27T08:48:42.870 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:42.870 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:42.870 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:42.870 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:42.870 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:42.871 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 169, 2024-05-27T08:48:42.871 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:42.871 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:42.871 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:42.871 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-27T08:48:42.871 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 536486, 2024-05-27T08:48:42.872 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 3982, 2024-05-27T08:48:42.872 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:42.872 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-27T08:48:42.872 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:42.872 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-27T08:48:42.872 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.873 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-27T08:48:42.873 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-27T08:48:42.873 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-27T08:48:42.873 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 3069, 2024-05-27T08:48:42.873 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 9271035, 2024-05-27T08:48:42.873 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 33, 2024-05-27T08:48:42.874 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 3069, 2024-05-27T08:48:42.874 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-27T08:48:42.874 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3069, 2024-05-27T08:48:42.874 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 5.823831623, 2024-05-27T08:48:42.874 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.001897631 2024-05-27T08:48:42.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.875 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-27T08:48:42.875 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-27T08:48:42.875 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 3069, 2024-05-27T08:48:42.875 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 3417 2024-05-27T08:48:42.875 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.875 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 3069, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 209, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 2860, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 0, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 10, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 209, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 1674, 2024-05-27T08:48:42.876 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 149, 2024-05-27T08:48:42.877 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-27T08:48:42.877 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-27T08:48:42.877 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-27T08:48:42.877 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-05-27T08:48:42.877 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 782, 2024-05-27T08:48:42.877 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-27T08:48:42.877 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:42.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:42.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 0, 2024-05-27T08:48:42.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 20, 2024-05-27T08:48:42.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:42.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:42.878 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:42.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-27T08:48:42.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-27T08:48:42.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-27T08:48:42.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:42.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-27T08:48:42.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:42.879 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-27T08:48:42.880 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-27T08:48:42.880 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-27T08:48:42.880 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-27T08:48:42.880 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-27T08:48:42.880 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-27T08:48:42.880 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-27T08:48:42.880 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-27T08:48:42.881 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-27T08:48:42.881 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-27T08:48:42.881 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-27T08:48:42.881 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-27T08:48:42.881 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-27T08:48:42.881 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-27T08:48:42.882 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-27T08:48:42.882 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-27T08:48:42.882 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-27T08:48:42.882 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 94, 2024-05-27T08:48:42.882 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-27T08:48:42.882 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 11, 2024-05-27T08:48:42.883 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 12, 2024-05-27T08:48:42.883 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 12, 2024-05-27T08:48:42.883 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-27T08:48:42.883 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-27T08:48:42.883 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 449, 2024-05-27T08:48:42.883 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 0, 2024-05-27T08:48:42.883 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 124 2024-05-27T08:48:42.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.884 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-27T08:48:42.884 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 1, 2024-05-27T08:48:42.884 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:42.884 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-27T08:48:42.884 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-27T08:48:42.884 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.884 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-27T08:48:42.885 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 154 2024-05-27T08:48:42.885 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.885 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-27T08:48:42.885 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:42.885 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 6, 2024-05-27T08:48:42.885 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-27T08:48:42.885 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 6, 2024-05-27T08:48:42.885 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 768, 2024-05-27T08:48:42.886 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 768, 2024-05-27T08:48:42.886 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:42.886 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.886 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:42.886 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:42.886 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-27T08:48:42.886 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:42.886 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 79819, 2024-05-27T08:48:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 19874215, 2024-05-27T08:48:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 79819, 2024-05-27T08:48:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 79819, 2024-05-27T08:48:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 19874215, 2024-05-27T08:48:42.887 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:42.888 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.888 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.888 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.888 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.888 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.888 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:42.888 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:42.888 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-27T08:48:42.889 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:42.889 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:42.889 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:42.889 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:42.889 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:42.889 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 3069, 2024-05-27T08:48:42.890 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 9348699, 2024-05-27T08:48:42.890 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3069, 2024-05-27T08:48:42.890 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 9348699, 2024-05-27T08:48:42.890 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:42.890 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.890 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.890 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.891 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.891 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.891 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:42.891 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:42.891 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-27T08:48:42.891 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:42.892 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:42.892 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:42.892 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:42.892 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:42.892 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 3069, 2024-05-27T08:48:42.892 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 3069, 2024-05-27T08:48:42.892 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 3069, 2024-05-27T08:48:42.893 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 3069, 2024-05-27T08:48:42.893 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:42.893 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.893 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.893 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.893 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.893 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.894 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:42.894 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:42.894 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-27T08:48:42.894 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:42.894 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 768, 2024-05-27T08:48:42.894 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 77664, 2024-05-27T08:48:42.894 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:42.894 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 768, 2024-05-27T08:48:42.895 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:42.895 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:42.895 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 209, 2024-05-27T08:48:42.895 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 77664, 2024-05-27T08:48:42.895 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:42.895 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.895 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x56071edc0500": { 2024-05-27T08:48:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 2296, 2024-05-27T08:48:42.896 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-27T08:48:42.897 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:42.897 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 6155, 2024-05-27T08:48:42.897 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 9065975, 2024-05-27T08:48:42.897 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:42.897 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 6155, 2024-05-27T08:48:42.897 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:42.898 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:42.898 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 1465, 2024-05-27T08:48:42.898 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 9063679, 2024-05-27T08:48:42.898 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:42.898 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:42.898 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:42.899 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:42.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.899 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:42.899 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-27T08:48:43.579 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-27T08:48:43.579 INFO:tasks.check_counter:Daemon mds.h mds.imported=5 2024-05-27T08:48:43.579 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=331 2024-05-27T08:48:43.579 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=767 2024-05-27T08:48:43.580 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-27T08:48:43.580 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:43.580 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:43.580 DEBUG:tasks.check_counter:Getting stats from k 2024-05-27T08:48:43.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-27T08:48:43.818 INFO:teuthology.orchestra.run.smithi092.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.b/config 2024-05-27T08:48:43.940 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:43 smithi092 ceph-mon[36049]: pgmap v324: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 560 KiB/s wr, 42 op/s 2024-05-27T08:48:44.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:43 smithi179 ceph-mon[36481]: pgmap v324: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 560 KiB/s wr, 42 op/s 2024-05-27T08:48:44.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:43 smithi077 ceph-mon[32007]: pgmap v324: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 560 KiB/s wr, 42 op/s 2024-05-27T08:48:45.349 INFO:teuthology.orchestra.run.smithi092.stdout:{ 2024-05-27T08:48:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:45.349 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 339, 2024-05-27T08:48:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 314, 2024-05-27T08:48:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 6045006, 2024-05-27T08:48:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 166224, 2024-05-27T08:48:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-05-27T08:48:45.350 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 2, 2024-05-27T08:48:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.056027203, 2024-05-27T08:48:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.014646733, 2024-05-27T08:48:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.045041344, 2024-05-27T08:48:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.007008553, 2024-05-27T08:48:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 313, 2024-05-27T08:48:45.351 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.014397540, 2024-05-27T08:48:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000045998 2024-05-27T08:48:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.352 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 1833168, 2024-05-27T08:48:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 144912 2024-05-27T08:48:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 766, 2024-05-27T08:48:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 941, 2024-05-27T08:48:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 14284934, 2024-05-27T08:48:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 707005, 2024-05-27T08:48:45.353 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 8, 2024-05-27T08:48:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 6, 2024-05-27T08:48:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.084469439, 2024-05-27T08:48:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.025108946, 2024-05-27T08:48:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.027697178, 2024-05-27T08:48:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.021908265, 2024-05-27T08:48:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 938, 2024-05-27T08:48:45.354 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.040399508, 2024-05-27T08:48:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000043069 2024-05-27T08:48:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.355 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-27T08:48:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 464048 2024-05-27T08:48:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_messages": 1740, 2024-05-27T08:48:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages": 1740, 2024-05-27T08:48:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_bytes": 17289730, 2024-05-27T08:48:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_bytes": 552554, 2024-05-27T08:48:45.356 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_created_connections": 6, 2024-05-27T08:48:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_active_connections": 5, 2024-05-27T08:48:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_total_time": 0.163883720, 2024-05-27T08:48:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_send_time": 0.052206536, 2024-05-27T08:48:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_recv_time": 0.048139021, 2024-05-27T08:48:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_running_fast_dispatch_time": 0.049534051, 2024-05-27T08:48:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 1739, 2024-05-27T08:48:45.357 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.062381391, 2024-05-27T08:48:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000035871 2024-05-27T08:48:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.358 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T08:48:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T08:48:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "cct": { 2024-05-27T08:48:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "total_workers": 1, 2024-05-27T08:48:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "unhealthy_workers": 0 2024-05-27T08:48:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-MDSRank": { 2024-05-27T08:48:45.359 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-27T08:48:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-27T08:48:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 4349, 2024-05-27T08:48:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.108645599, 2024-05-27T08:48:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000024981 2024-05-27T08:48:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.360 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "queue_len": 0, 2024-05-27T08:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "complete_latency": { 2024-05-27T08:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.361 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "mds": { 2024-05-27T08:48:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "request": 0, 2024-05-27T08:48:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "reply": 0, 2024-05-27T08:48:45.362 INFO:teuthology.orchestra.run.smithi092.stdout: "reply_latency": { 2024-05-27T08:48:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.363 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_reply": 0, 2024-05-27T08:48:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "forward": 0, 2024-05-27T08:48:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:45.364 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_commit": 0, 2024-05-27T08:48:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_split": 0, 2024-05-27T08:48:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_merge": 0, 2024-05-27T08:48:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes": 3523, 2024-05-27T08:48:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_top": 2, 2024-05-27T08:48:45.365 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_bottom": 3521, 2024-05-27T08:48:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_pinned": 3520, 2024-05-27T08:48:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_expired": 0, 2024-05-27T08:48:45.366 INFO:teuthology.orchestra.run.smithi092.stdout: "inodes_with_caps": 0, 2024-05-27T08:48:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "caps": 0, 2024-05-27T08:48:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "subtrees": 5, 2024-05-27T08:48:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse": 0, 2024-05-27T08:48:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_hit": 0, 2024-05-27T08:48:45.367 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_forward": 0, 2024-05-27T08:48:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_discover": 0, 2024-05-27T08:48:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "traverse_lock": 0, 2024-05-27T08:48:45.368 INFO:teuthology.orchestra.run.smithi092.stdout: "load_cent": 0, 2024-05-27T08:48:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "q": 0, 2024-05-27T08:48:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "exported": 0, 2024-05-27T08:48:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "exported_inodes": 0, 2024-05-27T08:48:45.369 INFO:teuthology.orchestra.run.smithi092.stdout: "imported": 0, 2024-05-27T08:48:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "imported_inodes": 0, 2024-05-27T08:48:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:45.370 INFO:teuthology.orchestra.run.smithi092.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rfiles": 0, 2024-05-27T08:48:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rbytes": 0, 2024-05-27T08:48:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "root_rsnaps": 0, 2024-05-27T08:48:45.371 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:45.372 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:45.373 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:48:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T08:48:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_grant": 0, 2024-05-27T08:48:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps": 0, 2024-05-27T08:48:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_caps_dirty": 0, 2024-05-27T08:48:45.374 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_cap_release": 0, 2024-05-27T08:48:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "process_request_cap_release": 0 2024-05-27T08:48:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_cache": { 2024-05-27T08:48:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update": 0, 2024-05-27T08:48:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_update_receipt": 0, 2024-05-27T08:48:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_try_discover": 0, 2024-05-27T08:48:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_send_discover": 0, 2024-05-27T08:48:45.375 INFO:teuthology.orchestra.run.smithi092.stdout: "dir_handle_discover": 0, 2024-05-27T08:48:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays": 0, 2024-05-27T08:48:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_created": 0, 2024-05-27T08:48:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_enqueued": 0, 2024-05-27T08:48:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "strays_migrated": 0, 2024-05-27T08:48:45.376 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_started": 0, 2024-05-27T08:48:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "recovery_completed": 0, 2024-05-27T08:48:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:48:45.377 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:48:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_flush": 0, 2024-05-27T08:48:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: "ireq_inodestats": 0 2024-05-27T08:48:45.378 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_log": { 2024-05-27T08:48:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "evlrg": 0, 2024-05-27T08:48:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "evadd": 0, 2024-05-27T08:48:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "evex": 0, 2024-05-27T08:48:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "evtrm": 0, 2024-05-27T08:48:45.379 INFO:teuthology.orchestra.run.smithi092.stdout: "ev": 22463, 2024-05-27T08:48:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "evexg": 0, 2024-05-27T08:48:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "evexd": 0, 2024-05-27T08:48:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "segadd": 0, 2024-05-27T08:48:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "segex": 0, 2024-05-27T08:48:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "segtrm": 0, 2024-05-27T08:48:45.380 INFO:teuthology.orchestra.run.smithi092.stdout: "seg": 28, 2024-05-27T08:48:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "segmjr": 3, 2024-05-27T08:48:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "segexg": 0, 2024-05-27T08:48:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "segexd": 0, 2024-05-27T08:48:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "expos": 4194304, 2024-05-27T08:48:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "wrpos": 39258552, 2024-05-27T08:48:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "rdpos": 39257409, 2024-05-27T08:48:45.381 INFO:teuthology.orchestra.run.smithi092.stdout: "jlat": { 2024-05-27T08:48:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: "replayed": 22463 2024-05-27T08:48:45.382 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_mem": { 2024-05-27T08:48:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "ino": 125, 2024-05-27T08:48:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "ino+": 3503, 2024-05-27T08:48:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "ino-": 3378, 2024-05-27T08:48:45.383 INFO:teuthology.orchestra.run.smithi092.stdout: "dir": 24, 2024-05-27T08:48:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "dir+": 24, 2024-05-27T08:48:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "dir-": 0, 2024-05-27T08:48:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "dn": 3526, 2024-05-27T08:48:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "dn+": 3526, 2024-05-27T08:48:45.384 INFO:teuthology.orchestra.run.smithi092.stdout: "dn-": 0, 2024-05-27T08:48:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "cap": 0, 2024-05-27T08:48:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "cap+": 0, 2024-05-27T08:48:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "cap-": 0, 2024-05-27T08:48:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "rss": 54724, 2024-05-27T08:48:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: "heap": 215416 2024-05-27T08:48:45.385 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_server": { 2024-05-27T08:48:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_client_request": 0, 2024-05-27T08:48:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "dispatch_server_request": 0, 2024-05-27T08:48:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_request": 0, 2024-05-27T08:48:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_client_session": 0, 2024-05-27T08:48:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "handle_peer_request": 0, 2024-05-27T08:48:45.386 INFO:teuthology.orchestra.run.smithi092.stdout: "req_create_latency": { 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getattr_latency": { 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.387 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "req_link_latency": { 2024-05-27T08:48:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.388 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookup_latency": { 2024-05-27T08:48:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.389 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupino_latency": { 2024-05-27T08:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.390 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupname_latency": { 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.391 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.392 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "req_lssnap_latency": { 2024-05-27T08:48:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.393 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mkdir_latency": { 2024-05-27T08:48:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.394 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mknod_latency": { 2024-05-27T08:48:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.395 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "req_mksnap_latency": { 2024-05-27T08:48:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.396 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "req_open_latency": { 2024-05-27T08:48:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.397 INFO:teuthology.orchestra.run.smithi092.stdout: "req_readdir_latency": { 2024-05-27T08:48:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.398 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rename_latency": { 2024-05-27T08:48:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.399 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.400 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmdir_latency": { 2024-05-27T08:48:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.401 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.402 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:45.403 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.404 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setattr_latency": { 2024-05-27T08:48:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.405 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.406 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.407 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setlayout_latency": { 2024-05-27T08:48:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.408 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "req_setxattr_latency": { 2024-05-27T08:48:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.409 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "req_symlink_latency": { 2024-05-27T08:48:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.410 INFO:teuthology.orchestra.run.smithi092.stdout: "req_unlink_latency": { 2024-05-27T08:48:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:45.411 INFO:teuthology.orchestra.run.smithi092.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.412 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_sessions": { 2024-05-27T08:48:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "session_count": 2, 2024-05-27T08:48:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "session_add": 4, 2024-05-27T08:48:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "session_remove": 2, 2024-05-27T08:48:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_open": 2, 2024-05-27T08:48:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "sessions_stale": 0, 2024-05-27T08:48:45.413 INFO:teuthology.orchestra.run.smithi092.stdout: "total_load": 0, 2024-05-27T08:48:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "average_load": 0, 2024-05-27T08:48:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "avg_session_uptime": 3378, 2024-05-27T08:48:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "mempool": { 2024-05-27T08:48:45.414 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:45.415 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:45.416 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:45.417 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:45.418 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:45.419 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_items": 0, 2024-05-27T08:48:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:45.420 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_bytes": 228681, 2024-05-27T08:48:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_anon_items": 197, 2024-05-27T08:48:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_bytes": 0, 2024-05-27T08:48:45.421 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_items": 0, 2024-05-27T08:48:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_items": 169, 2024-05-27T08:48:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:45.422 INFO:teuthology.orchestra.run.smithi092.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "pgmap_items": 0, 2024-05-27T08:48:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_bytes": 2882593, 2024-05-27T08:48:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "mds_co_items": 19165, 2024-05-27T08:48:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_1_items": 0, 2024-05-27T08:48:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:45.423 INFO:teuthology.orchestra.run.smithi092.stdout: "unittest_2_items": 0 2024-05-27T08:48:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "objecter": { 2024-05-27T08:48:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "op_active": 0, 2024-05-27T08:48:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "op_laggy": 0, 2024-05-27T08:48:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send": 2567, 2024-05-27T08:48:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "op_send_bytes": 21, 2024-05-27T08:48:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "op_resend": 3, 2024-05-27T08:48:45.424 INFO:teuthology.orchestra.run.smithi092.stdout: "op_reply": 2567, 2024-05-27T08:48:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "op_latency": { 2024-05-27T08:48:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2567, 2024-05-27T08:48:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 1.372960326, 2024-05-27T08:48:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000534850 2024-05-27T08:48:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "op_inflight": 0, 2024-05-27T08:48:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "oplen_avg": { 2024-05-27T08:48:45.425 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 2567, 2024-05-27T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 2568 2024-05-27T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "op": 2567, 2024-05-27T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "op_r": 2567, 2024-05-27T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "op_w": 0, 2024-05-27T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "op_rmw": 0, 2024-05-27T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "op_pg": 0, 2024-05-27T08:48:45.426 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_stat": 1503, 2024-05-27T08:48:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_create": 0, 2024-05-27T08:48:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_read": 1062, 2024-05-27T08:48:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_write": 0, 2024-05-27T08:48:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writefull": 0, 2024-05-27T08:48:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_writesame": 0, 2024-05-27T08:48:45.427 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_append": 0, 2024-05-27T08:48:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_zero": 0, 2024-05-27T08:48:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_truncate": 0, 2024-05-27T08:48:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_delete": 0, 2024-05-27T08:48:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_mapext": 0, 2024-05-27T08:48:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:45.428 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_getxattr": 1, 2024-05-27T08:48:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_setxattr": 0, 2024-05-27T08:48:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_call": 0, 2024-05-27T08:48:45.429 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_watch": 0, 2024-05-27T08:48:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_notify": 0, 2024-05-27T08:48:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls": 0, 2024-05-27T08:48:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "osdop_other": 0, 2024-05-27T08:48:45.430 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_active": 0, 2024-05-27T08:48:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_send": 0, 2024-05-27T08:48:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_resend": 0, 2024-05-27T08:48:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "linger_ping": 0, 2024-05-27T08:48:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_active": 0, 2024-05-27T08:48:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_send": 0, 2024-05-27T08:48:45.431 INFO:teuthology.orchestra.run.smithi092.stdout: "poolop_resend": 0, 2024-05-27T08:48:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_active": 0, 2024-05-27T08:48:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_send": 0, 2024-05-27T08:48:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "poolstat_resend": 0, 2024-05-27T08:48:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_active": 0, 2024-05-27T08:48:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_send": 0, 2024-05-27T08:48:45.432 INFO:teuthology.orchestra.run.smithi092.stdout: "statfs_resend": 0, 2024-05-27T08:48:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "command_active": 0, 2024-05-27T08:48:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "command_send": 0, 2024-05-27T08:48:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "command_resend": 0, 2024-05-27T08:48:45.433 INFO:teuthology.orchestra.run.smithi092.stdout: "map_epoch": 94, 2024-05-27T08:48:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "map_full": 0, 2024-05-27T08:48:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "map_inc": 11, 2024-05-27T08:48:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_sessions": 9, 2024-05-27T08:48:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_open": 9, 2024-05-27T08:48:45.434 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_session_close": 0, 2024-05-27T08:48:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "osd_laggy": 0, 2024-05-27T08:48:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_wr": 0, 2024-05-27T08:48:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_rd": 2, 2024-05-27T08:48:45.435 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_del": 0 2024-05-27T08:48:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "oft": { 2024-05-27T08:48:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_objs": 0, 2024-05-27T08:48:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:45.436 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_updates": 0, 2024-05-27T08:48:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "omap_total_removes": 0 2024-05-27T08:48:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "osd-slow-ops": { 2024-05-27T08:48:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: "slow_ops_count": 0 2024-05-27T08:48:45.437 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "purge_queue": { 2024-05-27T08:48:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T08:48:45.438 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing": 0, 2024-05-27T08:48:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executing_high_water": 0, 2024-05-27T08:48:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed_ops": 0, 2024-05-27T08:48:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_executed": 0, 2024-05-27T08:48:45.439 INFO:teuthology.orchestra.run.smithi092.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:45.440 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-27T08:48:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:45.441 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 2841, 2024-05-27T08:48:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 37347054, 2024-05-27T08:48:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 2841, 2024-05-27T08:48:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:45.442 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2841, 2024-05-27T08:48:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 37347054, 2024-05-27T08:48:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.443 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 104857600, 2024-05-27T08:48:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:45.444 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2567, 2024-05-27T08:48:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 35065424, 2024-05-27T08:48:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2567, 2024-05-27T08:48:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 35065424, 2024-05-27T08:48:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:45.445 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 1024, 2024-05-27T08:48:45.446 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 2567, 2024-05-27T08:48:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 2567, 2024-05-27T08:48:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 2567, 2024-05-27T08:48:45.447 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 2567, 2024-05-27T08:48:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.448 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-27T08:48:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:45.449 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-27T08:48:45.450 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-27T08:48:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.451 INFO:teuthology.orchestra.run.smithi092.stdout: }, 2024-05-27T08:48:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "throttle-write_buf_throttle-0x56431a2406e0": { 2024-05-27T08:48:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "val": 0, 2024-05-27T08:48:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "max": 3758096384, 2024-05-27T08:48:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "get_started": 0, 2024-05-27T08:48:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "get": 0, 2024-05-27T08:48:45.452 INFO:teuthology.orchestra.run.smithi092.stdout: "get_sum": 0, 2024-05-27T08:48:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "take": 0, 2024-05-27T08:48:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "take_sum": 0, 2024-05-27T08:48:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "put": 0, 2024-05-27T08:48:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "put_sum": 0, 2024-05-27T08:48:45.453 INFO:teuthology.orchestra.run.smithi092.stdout: "wait": { 2024-05-27T08:48:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "avgcount": 0, 2024-05-27T08:48:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "sum": 0.000000000, 2024-05-27T08:48:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: "avgtime": 0.000000000 2024-05-27T08:48:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.454 INFO:teuthology.orchestra.run.smithi092.stdout: } 2024-05-27T08:48:45.454 INFO:teuthology.orchestra.run.smithi092.stdout:} 2024-05-27T08:48:46.087 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:45 smithi092 ceph-mon[36049]: pgmap v325: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 287 KiB/s wr, 25 op/s 2024-05-27T08:48:46.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:45 smithi179 ceph-mon[36481]: pgmap v325: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 287 KiB/s wr, 25 op/s 2024-05-27T08:48:46.114 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-27T08:48:46.114 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-27T08:48:46.114 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-05-27T08:48:46.114 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-05-27T08:48:46.114 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-05-27T08:48:46.114 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:46.114 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:46.115 DEBUG:tasks.check_counter:Getting stats from c 2024-05-27T08:48:46.115 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-27T08:48:46.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:45 smithi077 ceph-mon[32007]: pgmap v325: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 287 KiB/s wr, 25 op/s 2024-05-27T08:48:46.367 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:48:48.024 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:47 smithi179 ceph-mon[36481]: pgmap v326: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 287 KiB/s wr, 25 op/s 2024-05-27T08:48:48.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:47 smithi077 ceph-mon[32007]: pgmap v326: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 287 KiB/s wr, 25 op/s 2024-05-27T08:48:48.153 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-27T08:48:48.153 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:48.153 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 78394, 2024-05-27T08:48:48.153 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 62622, 2024-05-27T08:48:48.154 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 23697233, 2024-05-27T08:48:48.154 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 31664991, 2024-05-27T08:48:48.154 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 11, 2024-05-27T08:48:48.154 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-05-27T08:48:48.154 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 3.553414427, 2024-05-27T08:48:48.154 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 1.387977923, 2024-05-27T08:48:48.154 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 2.615298520, 2024-05-27T08:48:48.155 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.116610465, 2024-05-27T08:48:48.155 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:48.157 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 62625, 2024-05-27T08:48:48.157 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.010595976, 2024-05-27T08:48:48.157 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000016137 2024-05-27T08:48:48.157 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.157 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:48.157 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 131122, 2024-05-27T08:48:48.158 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.019193288, 2024-05-27T08:48:48.158 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000146 2024-05-27T08:48:48.158 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.158 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-27T08:48:48.158 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 448 2024-05-27T08:48:48.158 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.158 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:48.159 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 37392, 2024-05-27T08:48:48.159 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 53872, 2024-05-27T08:48:48.159 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 13133504, 2024-05-27T08:48:48.159 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 51627520, 2024-05-27T08:48:48.159 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 16, 2024-05-27T08:48:48.159 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 9, 2024-05-27T08:48:48.159 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 2.227144192, 2024-05-27T08:48:48.159 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 1.016980567, 2024-05-27T08:48:48.160 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 1.406792376, 2024-05-27T08:48:48.160 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.101764656, 2024-05-27T08:48:48.160 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:48.160 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 53865, 2024-05-27T08:48:48.160 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.993464067, 2024-05-27T08:48:48.160 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000018443 2024-05-27T08:48:48.160 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.160 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:48.161 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 77361, 2024-05-27T08:48:48.161 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.009901652, 2024-05-27T08:48:48.162 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000127 2024-05-27T08:48:48.162 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.162 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 12060128, 2024-05-27T08:48:48.162 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 37250128 2024-05-27T08:48:48.162 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.162 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:48.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 4295, 2024-05-27T08:48:48.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 4420, 2024-05-27T08:48:48.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 3323892, 2024-05-27T08:48:48.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 7911540, 2024-05-27T08:48:48.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 14, 2024-05-27T08:48:48.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2024-05-27T08:48:48.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.316945784, 2024-05-27T08:48:48.164 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.121700148, 2024-05-27T08:48:48.164 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.135578380, 2024-05-27T08:48:48.164 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.068977653, 2024-05-27T08:48:48.164 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:48.164 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4419, 2024-05-27T08:48:48.164 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.126398550, 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000028603 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2480, 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000427034, 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000172 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1857504, 2024-05-27T08:48:48.165 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 464528 2024-05-27T08:48:48.166 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.166 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-05-27T08:48:48.166 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-05-27T08:48:48.166 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-05-27T08:48:48.166 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.166 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-05-27T08:48:48.167 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-27T08:48:48.167 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-27T08:48:48.167 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 14082, 2024-05-27T08:48:48.167 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 7.395628547, 2024-05-27T08:48:48.167 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000525183 2024-05-27T08:48:48.167 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.167 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.167 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:48.168 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-27T08:48:48.168 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-27T08:48:48.168 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 6722, 2024-05-27T08:48:48.168 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.347727118, 2024-05-27T08:48:48.168 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000051729 2024-05-27T08:48:48.168 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.169 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.169 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-05-27T08:48:48.169 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 22244, 2024-05-27T08:48:48.169 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 20027, 2024-05-27T08:48:48.169 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-05-27T08:48:48.169 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 20027, 2024-05-27T08:48:48.169 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 91.252925637, 2024-05-27T08:48:48.170 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.004556495 2024-05-27T08:48:48.170 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.170 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-05-27T08:48:48.170 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 2217, 2024-05-27T08:48:48.171 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:48.171 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:48.171 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-05-27T08:48:48.171 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-05-27T08:48:48.171 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-05-27T08:48:48.171 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 164, 2024-05-27T08:48:48.172 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 13, 2024-05-27T08:48:48.172 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 151, 2024-05-27T08:48:48.172 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:48.172 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 145, 2024-05-27T08:48:48.172 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-05-27T08:48:48.173 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 3, 2024-05-27T08:48:48.173 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 3, 2024-05-27T08:48:48.173 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 5, 2024-05-27T08:48:48.173 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 53025, 2024-05-27T08:48:48.173 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 25366, 2024-05-27T08:48:48.174 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 2217, 2024-05-27T08:48:48.174 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 27, 2024-05-27T08:48:48.174 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:48.174 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:48.174 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 1157, 2024-05-27T08:48:48.175 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 1276, 2024-05-27T08:48:48.175 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-05-27T08:48:48.175 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-05-27T08:48:48.176 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-05-27T08:48:48.176 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 3, 2024-05-27T08:48:48.176 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 38, 2024-05-27T08:48:48.176 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:48.176 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:48.176 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:48.177 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 67, 2024-05-27T08:48:48.177 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 21176629, 2024-05-27T08:48:48.177 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-05-27T08:48:48.177 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:48.177 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:48.177 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:48.178 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:48.178 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:48.178 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:48.178 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:48.179 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:48.179 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 4, 2024-05-27T08:48:48.179 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 441, 2024-05-27T08:48:48.179 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 13594, 2024-05-27T08:48:48.179 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:48.179 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:48.180 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:48.180 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 7335, 2024-05-27T08:48:48.180 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 4298, 2024-05-27T08:48:48.180 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 1408, 2024-05-27T08:48:48.180 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 19335 2024-05-27T08:48:48.180 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.180 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-05-27T08:48:48.181 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 527, 2024-05-27T08:48:48.181 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 797, 2024-05-27T08:48:48.181 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 405, 2024-05-27T08:48:48.181 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 812, 2024-05-27T08:48:48.181 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 602, 2024-05-27T08:48:48.181 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-05-27T08:48:48.182 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:48.182 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:48.182 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 3378, 2024-05-27T08:48:48.182 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 3378, 2024-05-27T08:48:48.182 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:48.182 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-05-27T08:48:48.183 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:48.183 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:48.183 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:48.183 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-05-27T08:48:48.183 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-05-27T08:48:48.184 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 7, 2024-05-27T08:48:48.184 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 205, 2024-05-27T08:48:48.184 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:48.184 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:48.184 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-05-27T08:48:48.184 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:48.185 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:48.185 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-05-27T08:48:48.185 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.185 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-05-27T08:48:48.185 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-05-27T08:48:48.185 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 22465, 2024-05-27T08:48:48.185 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-05-27T08:48:48.186 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-05-27T08:48:48.186 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 22465, 2024-05-27T08:48:48.186 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-05-27T08:48:48.186 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-05-27T08:48:48.186 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 28, 2024-05-27T08:48:48.186 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-05-27T08:48:48.187 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-05-27T08:48:48.187 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 28, 2024-05-27T08:48:48.187 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 3, 2024-05-27T08:48:48.187 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-05-27T08:48:48.187 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-05-27T08:48:48.187 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-05-27T08:48:48.188 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 39259700, 2024-05-27T08:48:48.188 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-05-27T08:48:48.188 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-05-27T08:48:48.188 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7051, 2024-05-27T08:48:48.188 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 11.959385838, 2024-05-27T08:48:48.188 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001696126 2024-05-27T08:48:48.188 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.189 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-05-27T08:48:48.189 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.189 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-05-27T08:48:48.189 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 132, 2024-05-27T08:48:48.189 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 3510, 2024-05-27T08:48:48.189 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 3378, 2024-05-27T08:48:48.189 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 27, 2024-05-27T08:48:48.190 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 433, 2024-05-27T08:48:48.190 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 406, 2024-05-27T08:48:48.190 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 164, 2024-05-27T08:48:48.190 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 4701, 2024-05-27T08:48:48.190 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 4537, 2024-05-27T08:48:48.190 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 3, 2024-05-27T08:48:48.190 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 3532, 2024-05-27T08:48:48.190 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 3529, 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 65412, 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223608 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 43796, 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 2, 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 22244, 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 1200, 2024-05-27T08:48:48.191 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 8011, 2024-05-27T08:48:48.192 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-05-27T08:48:48.192 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3366, 2024-05-27T08:48:48.192 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 41.083236254, 2024-05-27T08:48:48.192 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.012205358 2024-05-27T08:48:48.192 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.192 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-05-27T08:48:48.192 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 102, 2024-05-27T08:48:48.192 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.098234316, 2024-05-27T08:48:48.193 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000963081 2024-05-27T08:48:48.193 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.193 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:48.193 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.193 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.193 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.193 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.193 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-05-27T08:48:48.194 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.194 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.194 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.194 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.194 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-05-27T08:48:48.194 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5997, 2024-05-27T08:48:48.194 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.761471086, 2024-05-27T08:48:48.194 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000627225 2024-05-27T08:48:48.195 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.195 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:48.195 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.195 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.195 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.195 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.195 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-05-27T08:48:48.195 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.196 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.197 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.198 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.198 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-05-27T08:48:48.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.199 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.199 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.199 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-05-27T08:48:48.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.199 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.200 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.200 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-05-27T08:48:48.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.200 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.200 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.200 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-05-27T08:48:48.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.201 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-05-27T08:48:48.202 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5664, 2024-05-27T08:48:48.202 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.498483102, 2024-05-27T08:48:48.202 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000441116 2024-05-27T08:48:48.202 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.202 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-05-27T08:48:48.202 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1174, 2024-05-27T08:48:48.202 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 37.158029585, 2024-05-27T08:48:48.203 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.031650791 2024-05-27T08:48:48.203 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.203 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:48.203 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.203 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.204 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.204 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.204 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:48.204 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.204 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.204 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.205 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.205 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-05-27T08:48:48.205 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.205 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.205 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.205 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.205 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:48.206 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.206 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.206 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.206 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.206 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:48.207 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.207 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.207 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.207 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.207 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-05-27T08:48:48.207 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 34, 2024-05-27T08:48:48.207 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.085303663, 2024-05-27T08:48:48.207 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002508931 2024-05-27T08:48:48.208 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.208 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:48.208 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.208 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.208 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.208 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.208 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:48.208 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 312, 2024-05-27T08:48:48.209 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.141196742, 2024-05-27T08:48:48.209 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000452553 2024-05-27T08:48:48.209 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.209 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-05-27T08:48:48.209 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.209 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.209 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.209 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.210 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-05-27T08:48:48.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.210 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.210 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.210 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-05-27T08:48:48.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.210 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.211 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.211 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-05-27T08:48:48.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3378, 2024-05-27T08:48:48.211 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 6.426970889, 2024-05-27T08:48:48.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001902596 2024-05-27T08:48:48.211 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.211 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:48.212 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:48.212 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:48.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.212 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.212 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.212 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.212 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-05-27T08:48:48.213 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 2, 2024-05-27T08:48:48.213 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 4, 2024-05-27T08:48:48.213 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-05-27T08:48:48.213 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 2, 2024-05-27T08:48:48.213 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-05-27T08:48:48.213 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 3633, 2024-05-27T08:48:48.213 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1816, 2024-05-27T08:48:48.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 554, 2024-05-27T08:48:48.214 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:48.214 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.214 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-05-27T08:48:48.214 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:48.214 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:48.214 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:48.214 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:48.215 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:48.215 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:48.215 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:48.215 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:48.215 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:48.215 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:48.215 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:48.215 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:48.216 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:48.216 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:48.216 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:48.216 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:48.216 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:48.216 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:48.216 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:48.216 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:48.217 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:48.217 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:48.217 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:48.217 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:48.217 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:48.217 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:48.217 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:48.217 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:48.218 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:48.218 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:48.218 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:48.218 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-05-27T08:48:48.218 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:48.218 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:48.218 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:48.219 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:48.219 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 302046, 2024-05-27T08:48:48.219 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 155, 2024-05-27T08:48:48.219 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:48.219 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:48.219 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-05-27T08:48:48.219 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-05-27T08:48:48.219 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 169, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:48.220 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-05-27T08:48:48.221 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 498256, 2024-05-27T08:48:48.221 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 3319, 2024-05-27T08:48:48.221 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:48.221 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-05-27T08:48:48.221 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:48.221 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-05-27T08:48:48.221 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.221 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-05-27T08:48:48.222 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-05-27T08:48:48.222 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-05-27T08:48:48.222 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 11447, 2024-05-27T08:48:48.222 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 35805048, 2024-05-27T08:48:48.222 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 33, 2024-05-27T08:48:48.222 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 11447, 2024-05-27T08:48:48.222 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-05-27T08:48:48.223 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 11447, 2024-05-27T08:48:48.223 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 21.959513507, 2024-05-27T08:48:48.223 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001918364 2024-05-27T08:48:48.223 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.223 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-05-27T08:48:48.223 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-05-27T08:48:48.223 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 11447, 2024-05-27T08:48:48.224 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 11863 2024-05-27T08:48:48.224 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.224 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 11447, 2024-05-27T08:48:48.224 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 265, 2024-05-27T08:48:48.224 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 11182, 2024-05-27T08:48:48.224 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-05-27T08:48:48.224 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-05-27T08:48:48.224 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-05-27T08:48:48.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-05-27T08:48:48.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 265, 2024-05-27T08:48:48.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 7316, 2024-05-27T08:48:48.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 174, 2024-05-27T08:48:48.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-05-27T08:48:48.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-05-27T08:48:48.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-05-27T08:48:48.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-05-27T08:48:48.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 3398, 2024-05-27T08:48:48.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-05-27T08:48:48.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:48.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:48.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-05-27T08:48:48.227 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-05-27T08:48:48.227 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:48.227 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:48.227 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:48.227 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-05-27T08:48:48.227 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-05-27T08:48:48.228 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-05-27T08:48:48.229 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-05-27T08:48:48.229 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-05-27T08:48:48.229 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-05-27T08:48:48.229 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-05-27T08:48:48.229 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-05-27T08:48:48.229 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-05-27T08:48:48.230 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-05-27T08:48:48.230 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-05-27T08:48:48.230 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-05-27T08:48:48.230 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-05-27T08:48:48.230 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-05-27T08:48:48.230 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-05-27T08:48:48.231 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-05-27T08:48:48.231 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-05-27T08:48:48.231 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 11, 2024-05-27T08:48:48.231 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-05-27T08:48:48.231 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-05-27T08:48:48.231 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-05-27T08:48:48.231 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-05-27T08:48:48.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 500, 2024-05-27T08:48:48.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-05-27T08:48:48.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 180 2024-05-27T08:48:48.232 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.232 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-05-27T08:48:48.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-05-27T08:48:48.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:48.233 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-05-27T08:48:48.233 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-05-27T08:48:48.233 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.233 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-05-27T08:48:48.233 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 215 2024-05-27T08:48:48.233 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.234 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-05-27T08:48:48.234 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:48.234 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 6, 2024-05-27T08:48:48.234 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-05-27T08:48:48.234 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 6, 2024-05-27T08:48:48.235 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 3378, 2024-05-27T08:48:48.235 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 3378, 2024-05-27T08:48:48.235 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:48.235 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.235 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:48.235 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:48.235 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 120077, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 27819525, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 120077, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 120077, 2024-05-27T08:48:48.236 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 27819525, 2024-05-27T08:48:48.237 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:48.237 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.237 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.237 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.237 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.237 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.237 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:48.237 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:48.238 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-27T08:48:48.238 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:48.238 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:48.238 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:48.238 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:48.238 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:48.238 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 11447, 2024-05-27T08:48:48.238 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 36146410, 2024-05-27T08:48:48.239 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 11447, 2024-05-27T08:48:48.239 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 36146410, 2024-05-27T08:48:48.239 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:48.239 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.239 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.239 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.239 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.239 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.240 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:48.240 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:48.240 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-05-27T08:48:48.240 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:48.240 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:48.240 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:48.240 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:48.240 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:48.241 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 11447, 2024-05-27T08:48:48.241 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 11447, 2024-05-27T08:48:48.241 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 11447, 2024-05-27T08:48:48.241 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 11447, 2024-05-27T08:48:48.241 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:48.241 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.241 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.241 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.242 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.242 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.242 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:48.242 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:48.242 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-27T08:48:48.242 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:48.242 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 3378, 2024-05-27T08:48:48.242 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 341362, 2024-05-27T08:48:48.243 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:48.243 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 3378, 2024-05-27T08:48:48.243 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:48.243 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:48.243 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 265, 2024-05-27T08:48:48.243 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 341362, 2024-05-27T08:48:48.243 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:48.243 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55ccfabd0500": { 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 574, 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:48.244 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 22465, 2024-05-27T08:48:48.245 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 35065396, 2024-05-27T08:48:48.245 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:48.245 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 22465, 2024-05-27T08:48:48.245 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:48.245 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:48.245 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 7051, 2024-05-27T08:48:48.245 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 35064822, 2024-05-27T08:48:48.245 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:48.246 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:48.246 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:48.246 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:48.246 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.246 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:48.246 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-27T08:48:48.247 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:47 smithi092 ceph-mon[36049]: pgmap v326: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 287 KiB/s wr, 25 op/s 2024-05-27T08:48:48.855 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-27T08:48:48.855 INFO:tasks.check_counter:Daemon mds.c mds.imported=3 2024-05-27T08:48:48.855 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=527 2024-05-27T08:48:48.855 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=797 2024-05-27T08:48:48.855 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-27T08:48:48.856 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:48.856 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:48.856 DEBUG:tasks.check_counter:Getting stats from f 2024-05-27T08:48:48.856 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-27T08:48:49.093 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:48:49.855 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:49 smithi179 ceph-mon[36481]: pgmap v327: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 298 KiB/s wr, 31 op/s 2024-05-27T08:48:50.129 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:49 smithi077 ceph-mon[32007]: pgmap v327: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 298 KiB/s wr, 31 op/s 2024-05-27T08:48:50.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:49 smithi092 ceph-mon[36049]: pgmap v327: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 298 KiB/s wr, 31 op/s 2024-05-27T08:48:50.899 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-27T08:48:50.899 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:50.899 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 67866, 2024-05-27T08:48:50.899 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 40316, 2024-05-27T08:48:50.899 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 21305404, 2024-05-27T08:48:50.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 15570821, 2024-05-27T08:48:50.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 11, 2024-05-27T08:48:50.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-05-27T08:48:50.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 2.913121936, 2024-05-27T08:48:50.900 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 1.121988423, 2024-05-27T08:48:50.901 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 2.308565651, 2024-05-27T08:48:50.901 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.057327035, 2024-05-27T08:48:50.901 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:50.901 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 40316, 2024-05-27T08:48:50.901 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.661992155, 2024-05-27T08:48:50.901 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000016420 2024-05-27T08:48:50.901 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.901 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:50.902 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 103206, 2024-05-27T08:48:50.902 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.015004979, 2024-05-27T08:48:50.902 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000145 2024-05-27T08:48:50.902 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.902 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:48:50.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:48:50.903 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.903 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:50.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 16952, 2024-05-27T08:48:50.903 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 22621, 2024-05-27T08:48:50.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 6006847, 2024-05-27T08:48:50.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 23282173, 2024-05-27T08:48:50.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 16, 2024-05-27T08:48:50.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 12, 2024-05-27T08:48:50.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 1.165540736, 2024-05-27T08:48:50.904 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.513478532, 2024-05-27T08:48:50.905 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.820090010, 2024-05-27T08:48:50.905 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.024486997, 2024-05-27T08:48:50.905 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:50.905 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 22615, 2024-05-27T08:48:50.905 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.437332362, 2024-05-27T08:48:50.906 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000019338 2024-05-27T08:48:50.906 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.906 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:50.906 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 34612, 2024-05-27T08:48:50.906 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.004666853, 2024-05-27T08:48:50.906 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000134 2024-05-27T08:48:50.907 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.908 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 5738464, 2024-05-27T08:48:50.908 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 18715968 2024-05-27T08:48:50.908 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.908 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:50.908 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1608, 2024-05-27T08:48:50.908 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1696, 2024-05-27T08:48:50.908 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 2339286, 2024-05-27T08:48:50.908 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 1023926, 2024-05-27T08:48:50.909 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 13, 2024-05-27T08:48:50.909 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 6, 2024-05-27T08:48:50.909 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.139604063, 2024-05-27T08:48:50.909 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.054241500, 2024-05-27T08:48:50.909 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.089770598, 2024-05-27T08:48:50.909 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.013048177, 2024-05-27T08:48:50.909 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:50.909 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1697, 2024-05-27T08:48:50.910 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.056931256, 2024-05-27T08:48:50.910 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000033548 2024-05-27T08:48:50.910 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.910 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:50.910 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1886, 2024-05-27T08:48:50.910 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000378466, 2024-05-27T08:48:50.910 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000200 2024-05-27T08:48:50.911 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.911 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1851920, 2024-05-27T08:48:50.911 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 465072 2024-05-27T08:48:50.911 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.911 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-05-27T08:48:50.912 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-05-27T08:48:50.912 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-05-27T08:48:50.912 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.912 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-05-27T08:48:50.912 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-27T08:48:50.912 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-27T08:48:50.912 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4070, 2024-05-27T08:48:50.912 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.102747874, 2024-05-27T08:48:50.913 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000516645 2024-05-27T08:48:50.913 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.913 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.913 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:50.913 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-27T08:48:50.913 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-27T08:48:50.913 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2247, 2024-05-27T08:48:50.913 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.098666937, 2024-05-27T08:48:50.914 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000043910 2024-05-27T08:48:50.914 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.914 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.914 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-05-27T08:48:50.914 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 11424, 2024-05-27T08:48:50.914 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 8478, 2024-05-27T08:48:50.914 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-05-27T08:48:50.914 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 8478, 2024-05-27T08:48:50.915 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 7.976798828, 2024-05-27T08:48:50.915 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000940882 2024-05-27T08:48:50.915 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.915 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-05-27T08:48:50.915 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 2946, 2024-05-27T08:48:50.915 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:50.915 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:50.915 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-05-27T08:48:50.916 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-05-27T08:48:50.916 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-05-27T08:48:50.916 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 188, 2024-05-27T08:48:50.916 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 36, 2024-05-27T08:48:50.916 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 152, 2024-05-27T08:48:50.916 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:50.916 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 136, 2024-05-27T08:48:50.916 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-05-27T08:48:50.917 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 6, 2024-05-27T08:48:50.917 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 6, 2024-05-27T08:48:50.917 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 10, 2024-05-27T08:48:50.917 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 21603, 2024-05-27T08:48:50.917 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 11403, 2024-05-27T08:48:50.917 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 2946, 2024-05-27T08:48:50.917 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 43, 2024-05-27T08:48:50.918 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:50.918 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:50.918 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 474, 2024-05-27T08:48:50.918 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 390, 2024-05-27T08:48:50.918 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-05-27T08:48:50.918 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-05-27T08:48:50.918 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-05-27T08:48:50.918 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 4, 2024-05-27T08:48:50.919 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 70, 2024-05-27T08:48:50.919 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:50.919 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:50.919 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:50.919 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 67, 2024-05-27T08:48:50.919 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 21176629, 2024-05-27T08:48:50.919 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-05-27T08:48:50.919 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:50.920 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:50.920 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:50.920 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:50.920 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:50.920 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:50.920 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:50.920 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:50.921 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:48:50.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 1303, 2024-05-27T08:48:50.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 5269, 2024-05-27T08:48:50.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:50.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:50.921 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:50.921 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 3219, 2024-05-27T08:48:50.921 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 734, 2024-05-27T08:48:50.922 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 639, 2024-05-27T08:48:50.922 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 8262 2024-05-27T08:48:50.922 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.922 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-05-27T08:48:50.922 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 322, 2024-05-27T08:48:50.922 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 733, 2024-05-27T08:48:50.922 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 373, 2024-05-27T08:48:50.922 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 780, 2024-05-27T08:48:50.923 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 330, 2024-05-27T08:48:50.923 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-05-27T08:48:50.923 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:50.923 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:50.923 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 822, 2024-05-27T08:48:50.923 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 822, 2024-05-27T08:48:50.923 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:50.923 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-05-27T08:48:50.924 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:50.924 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:50.924 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:50.924 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-05-27T08:48:50.924 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-05-27T08:48:50.924 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 7, 2024-05-27T08:48:50.924 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 189, 2024-05-27T08:48:50.924 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:50.925 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:50.925 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-05-27T08:48:50.925 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:50.925 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:50.925 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-05-27T08:48:50.925 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.925 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-05-27T08:48:50.925 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-05-27T08:48:50.926 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 7361, 2024-05-27T08:48:50.926 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-05-27T08:48:50.926 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-05-27T08:48:50.926 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 7361, 2024-05-27T08:48:50.926 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-05-27T08:48:50.926 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-05-27T08:48:50.926 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 10, 2024-05-27T08:48:50.926 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-05-27T08:48:50.927 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-05-27T08:48:50.927 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 10, 2024-05-27T08:48:50.927 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-05-27T08:48:50.927 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-05-27T08:48:50.927 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-05-27T08:48:50.927 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-05-27T08:48:50.927 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 14771275, 2024-05-27T08:48:50.927 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-05-27T08:48:50.928 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-05-27T08:48:50.928 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1946, 2024-05-27T08:48:50.928 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.495821233, 2024-05-27T08:48:50.928 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001796413 2024-05-27T08:48:50.928 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.928 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-05-27T08:48:50.929 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.929 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-05-27T08:48:50.929 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 172, 2024-05-27T08:48:50.929 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 994, 2024-05-27T08:48:50.929 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 822, 2024-05-27T08:48:50.929 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 32, 2024-05-27T08:48:50.929 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 417, 2024-05-27T08:48:50.930 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 385, 2024-05-27T08:48:50.930 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 188, 2024-05-27T08:48:50.930 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 1768, 2024-05-27T08:48:50.930 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 1580, 2024-05-27T08:48:50.930 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 6, 2024-05-27T08:48:50.930 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 1090, 2024-05-27T08:48:50.930 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 1084, 2024-05-27T08:48:50.931 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 60600, 2024-05-27T08:48:50.931 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223608 2024-05-27T08:48:50.931 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.931 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-05-27T08:48:50.931 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 17370, 2024-05-27T08:48:50.931 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-05-27T08:48:50.932 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 11424, 2024-05-27T08:48:50.932 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 22, 2024-05-27T08:48:50.932 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 2119, 2024-05-27T08:48:50.932 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-05-27T08:48:50.932 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 822, 2024-05-27T08:48:50.932 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.850046993, 2024-05-27T08:48:50.932 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002250665 2024-05-27T08:48:50.932 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.933 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-05-27T08:48:50.933 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 92, 2024-05-27T08:48:50.933 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.141251501, 2024-05-27T08:48:50.933 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001535342 2024-05-27T08:48:50.933 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.933 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-05-27T08:48:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.934 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.935 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.935 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-05-27T08:48:50.935 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1893, 2024-05-27T08:48:50.935 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.330732704, 2024-05-27T08:48:50.935 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000702975 2024-05-27T08:48:50.935 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.936 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:50.936 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.936 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.936 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.936 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.936 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-05-27T08:48:50.936 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.937 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.937 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.937 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.937 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-05-27T08:48:50.937 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.938 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.938 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.938 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.938 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:50.938 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.938 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.938 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.939 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.939 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:50.939 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.939 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.939 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.939 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.940 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-05-27T08:48:50.941 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.941 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.941 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.941 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.941 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-05-27T08:48:50.941 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.941 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.942 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.942 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.942 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-05-27T08:48:50.942 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.942 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.942 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.942 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.943 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.944 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.944 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-05-27T08:48:50.944 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4375, 2024-05-27T08:48:50.944 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2.090602008, 2024-05-27T08:48:50.944 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000477851 2024-05-27T08:48:50.945 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.945 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-05-27T08:48:50.945 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 474, 2024-05-27T08:48:50.945 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.053467289, 2024-05-27T08:48:50.945 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002222504 2024-05-27T08:48:50.945 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.946 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:50.946 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.946 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.946 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.946 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.946 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:50.946 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.946 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.947 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.947 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.947 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-05-27T08:48:50.947 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.947 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.947 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.947 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.947 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:50.948 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.948 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.948 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.948 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.949 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:50.949 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.949 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.949 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.949 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.949 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-05-27T08:48:50.949 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.949 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.950 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.950 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.950 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:50.950 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.950 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.950 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.950 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.950 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:50.951 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.951 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.951 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.951 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.951 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-05-27T08:48:50.951 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.952 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.952 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.952 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.952 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-05-27T08:48:50.952 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.952 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.952 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.952 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.953 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-05-27T08:48:50.953 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.953 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.953 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.953 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-05-27T08:48:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 822, 2024-05-27T08:48:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.510698333, 2024-05-27T08:48:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001837832 2024-05-27T08:48:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:50.954 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:50.955 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.955 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.955 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.955 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.955 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.955 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-05-27T08:48:50.955 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 2, 2024-05-27T08:48:50.955 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 3, 2024-05-27T08:48:50.956 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 1, 2024-05-27T08:48:50.956 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 2, 2024-05-27T08:48:50.956 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-05-27T08:48:50.956 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 2085, 2024-05-27T08:48:50.956 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1042, 2024-05-27T08:48:50.957 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 557, 2024-05-27T08:48:50.957 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:50.957 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.957 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-05-27T08:48:50.957 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:50.957 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:50.957 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:50.957 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:50.958 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:50.958 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:50.958 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:50.958 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:50.958 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:50.958 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:50.958 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:50.959 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:50.959 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:50.959 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:50.959 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:50.959 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:50.959 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:50.959 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:50.959 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:50.960 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:50.960 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:50.960 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:50.960 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:50.960 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:50.960 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:50.960 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:50.960 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:50.961 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:50.961 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:50.961 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:50.961 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:50.961 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-05-27T08:48:50.961 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:50.961 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:50.961 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:50.962 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:50.962 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 283870, 2024-05-27T08:48:50.962 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 187, 2024-05-27T08:48:50.962 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:50.962 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:50.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-05-27T08:48:50.962 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-05-27T08:48:50.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:50.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:50.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:50.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:50.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:50.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 169, 2024-05-27T08:48:50.963 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:50.964 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:50.964 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:50.964 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-05-27T08:48:50.964 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 594022, 2024-05-27T08:48:50.964 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 4040, 2024-05-27T08:48:50.964 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:50.964 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-05-27T08:48:50.965 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:50.965 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-05-27T08:48:50.965 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.965 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-05-27T08:48:50.965 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-05-27T08:48:50.965 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-05-27T08:48:50.966 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 3688, 2024-05-27T08:48:50.966 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 10806605, 2024-05-27T08:48:50.966 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 33, 2024-05-27T08:48:50.966 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 3688, 2024-05-27T08:48:50.966 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-05-27T08:48:50.966 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3688, 2024-05-27T08:48:50.967 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 6.235861944, 2024-05-27T08:48:50.967 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001690851 2024-05-27T08:48:50.967 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.967 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-05-27T08:48:50.967 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-05-27T08:48:50.967 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3688, 2024-05-27T08:48:50.967 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 4061 2024-05-27T08:48:50.968 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.968 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 3688, 2024-05-27T08:48:50.968 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 234, 2024-05-27T08:48:50.968 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 3454, 2024-05-27T08:48:50.968 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-05-27T08:48:50.968 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-05-27T08:48:50.969 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-05-27T08:48:50.969 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-05-27T08:48:50.969 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 234, 2024-05-27T08:48:50.969 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 2180, 2024-05-27T08:48:50.969 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 168, 2024-05-27T08:48:50.970 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-05-27T08:48:50.970 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-05-27T08:48:50.970 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-05-27T08:48:50.970 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-05-27T08:48:50.970 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 836, 2024-05-27T08:48:50.970 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-05-27T08:48:50.970 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:50.971 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:50.971 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-05-27T08:48:50.971 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-05-27T08:48:50.971 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:50.971 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:50.971 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:50.972 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-05-27T08:48:50.972 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-05-27T08:48:50.972 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-05-27T08:48:50.972 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:50.972 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-05-27T08:48:50.972 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:50.972 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-05-27T08:48:50.973 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-05-27T08:48:50.973 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-05-27T08:48:50.973 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-05-27T08:48:50.973 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-05-27T08:48:50.973 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-05-27T08:48:50.974 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-05-27T08:48:50.974 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-05-27T08:48:50.974 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-05-27T08:48:50.974 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-05-27T08:48:50.974 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-05-27T08:48:50.974 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-05-27T08:48:50.974 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-05-27T08:48:50.975 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-05-27T08:48:50.975 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-05-27T08:48:50.975 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-05-27T08:48:50.976 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-05-27T08:48:50.976 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-05-27T08:48:50.976 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-05-27T08:48:50.976 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 11, 2024-05-27T08:48:50.976 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-05-27T08:48:50.976 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-05-27T08:48:50.977 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-05-27T08:48:50.977 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-05-27T08:48:50.977 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 468, 2024-05-27T08:48:50.977 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-05-27T08:48:50.977 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 145 2024-05-27T08:48:50.977 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.978 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-05-27T08:48:50.978 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-05-27T08:48:50.978 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:50.978 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-05-27T08:48:50.979 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-05-27T08:48:50.979 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.979 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-05-27T08:48:50.979 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 196 2024-05-27T08:48:50.979 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.979 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-05-27T08:48:50.980 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:50.980 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 6, 2024-05-27T08:48:50.980 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-05-27T08:48:50.980 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 6, 2024-05-27T08:48:50.980 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 822, 2024-05-27T08:48:50.980 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 822, 2024-05-27T08:48:50.981 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:50.981 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.981 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:50.981 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:50.981 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-27T08:48:50.981 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:50.981 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 86422, 2024-05-27T08:48:50.981 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 21148336, 2024-05-27T08:48:50.982 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:50.982 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 86422, 2024-05-27T08:48:50.982 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:50.982 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:50.982 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 86422, 2024-05-27T08:48:50.982 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 21148336, 2024-05-27T08:48:50.982 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:50.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.983 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.983 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.983 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.983 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.983 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:50.983 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:50.983 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-27T08:48:50.983 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3688, 2024-05-27T08:48:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 10889699, 2024-05-27T08:48:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3688, 2024-05-27T08:48:50.984 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 10889699, 2024-05-27T08:48:50.985 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:50.985 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.985 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.985 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.985 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.986 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.986 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:50.986 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:50.986 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-05-27T08:48:50.986 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:50.986 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:50.986 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:50.986 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:50.987 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:50.987 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3688, 2024-05-27T08:48:50.987 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 3688, 2024-05-27T08:48:50.987 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3688, 2024-05-27T08:48:50.987 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 3688, 2024-05-27T08:48:50.987 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:50.987 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.987 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 822, 2024-05-27T08:48:50.988 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 83094, 2024-05-27T08:48:50.989 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:50.989 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 822, 2024-05-27T08:48:50.989 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:50.989 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:50.989 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 234, 2024-05-27T08:48:50.989 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 83094, 2024-05-27T08:48:50.989 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:50.990 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.990 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.990 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.990 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.990 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:50.990 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x56121c1f0500": { 2024-05-27T08:48:50.990 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:50.991 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-27T08:48:50.991 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:50.991 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 7361, 2024-05-27T08:48:50.991 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 10576971, 2024-05-27T08:48:50.991 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:50.991 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 7361, 2024-05-27T08:48:50.991 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:50.991 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:50.992 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 1946, 2024-05-27T08:48:50.992 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 10576971, 2024-05-27T08:48:50.992 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:50.992 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:50.992 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:50.992 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:50.992 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.993 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:50.993 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-27T08:48:51.672 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-27T08:48:51.672 INFO:tasks.check_counter:Daemon mds.f mds.imported=4 2024-05-27T08:48:51.672 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=322 2024-05-27T08:48:51.672 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=733 2024-05-27T08:48:51.672 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-27T08:48:51.673 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:51.673 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:51.673 DEBUG:tasks.check_counter:Getting stats from i 2024-05-27T08:48:51.673 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-27T08:48:51.912 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:48:52.034 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:51 smithi179 ceph-mon[36481]: pgmap v328: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-27T08:48:52.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:51 smithi077 ceph-mon[32007]: pgmap v328: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-27T08:48:52.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:51 smithi092 ceph-mon[36049]: pgmap v328: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-05-27T08:48:53.643 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-27T08:48:53.643 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:53.643 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 76603, 2024-05-27T08:48:53.643 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 58614, 2024-05-27T08:48:53.643 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 23382358, 2024-05-27T08:48:53.643 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 30477302, 2024-05-27T08:48:53.644 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 11, 2024-05-27T08:48:53.644 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-05-27T08:48:53.644 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 3.485042284, 2024-05-27T08:48:53.644 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 1.372355997, 2024-05-27T08:48:53.644 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 2.559563494, 2024-05-27T08:48:53.644 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.118232338, 2024-05-27T08:48:53.644 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:53.644 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 58612, 2024-05-27T08:48:53.645 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.978336712, 2024-05-27T08:48:53.645 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000016691 2024-05-27T08:48:53.645 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.645 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:53.645 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 125777, 2024-05-27T08:48:53.645 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.019935769, 2024-05-27T08:48:53.645 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000158 2024-05-27T08:48:53.645 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:48:53.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:48:53.646 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.646 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:53.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 31097, 2024-05-27T08:48:53.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 50788, 2024-05-27T08:48:53.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 11063850, 2024-05-27T08:48:53.646 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 45472363, 2024-05-27T08:48:53.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 21, 2024-05-27T08:48:53.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 9, 2024-05-27T08:48:53.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 1.936525349, 2024-05-27T08:48:53.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.906881113, 2024-05-27T08:48:53.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 1.138787003, 2024-05-27T08:48:53.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.065687714, 2024-05-27T08:48:53.647 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:53.648 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 50786, 2024-05-27T08:48:53.648 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.810004429, 2024-05-27T08:48:53.648 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000015949 2024-05-27T08:48:53.648 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.648 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:53.648 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 70047, 2024-05-27T08:48:53.648 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.009016774, 2024-05-27T08:48:53.648 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000128 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 10498112, 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 40476000 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 3963, 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 3900, 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 3338582, 2024-05-27T08:48:53.649 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 6868225, 2024-05-27T08:48:53.650 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 8, 2024-05-27T08:48:53.650 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 7, 2024-05-27T08:48:53.650 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.300980730, 2024-05-27T08:48:53.650 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.113288641, 2024-05-27T08:48:53.650 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.129068196, 2024-05-27T08:48:53.650 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.068723383, 2024-05-27T08:48:53.650 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:53.651 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3899, 2024-05-27T08:48:53.651 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.109006948, 2024-05-27T08:48:53.651 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000027957 2024-05-27T08:48:53.651 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.651 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:53.651 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2261, 2024-05-27T08:48:53.651 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000420926, 2024-05-27T08:48:53.651 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000186 2024-05-27T08:48:53.652 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.652 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1860656, 2024-05-27T08:48:53.652 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 148768 2024-05-27T08:48:53.652 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.652 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-05-27T08:48:53.652 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-05-27T08:48:53.652 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-05-27T08:48:53.652 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.653 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-05-27T08:48:53.653 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-27T08:48:53.653 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-27T08:48:53.653 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 8945, 2024-05-27T08:48:53.653 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 5.390613881, 2024-05-27T08:48:53.653 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000602639 2024-05-27T08:48:53.653 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.654 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.654 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:53.654 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-27T08:48:53.654 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-27T08:48:53.654 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 7290, 2024-05-27T08:48:53.654 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.390164256, 2024-05-27T08:48:53.654 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000053520 2024-05-27T08:48:53.654 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.655 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.655 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-05-27T08:48:53.655 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 24139, 2024-05-27T08:48:53.655 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 21591, 2024-05-27T08:48:53.655 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-05-27T08:48:53.655 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 21591, 2024-05-27T08:48:53.655 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 22.885952017, 2024-05-27T08:48:53.655 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001059976 2024-05-27T08:48:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-05-27T08:48:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 2548, 2024-05-27T08:48:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-05-27T08:48:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-05-27T08:48:53.656 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 167, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 21, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 146, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 137, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 3, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 3, 2024-05-27T08:48:53.657 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 4, 2024-05-27T08:48:53.658 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 49905, 2024-05-27T08:48:53.658 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 19879, 2024-05-27T08:48:53.658 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 2545, 2024-05-27T08:48:53.658 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 39, 2024-05-27T08:48:53.658 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:53.658 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:53.658 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 315, 2024-05-27T08:48:53.659 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 1233, 2024-05-27T08:48:53.659 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-05-27T08:48:53.659 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-05-27T08:48:53.659 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-05-27T08:48:53.659 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 2, 2024-05-27T08:48:53.659 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 74, 2024-05-27T08:48:53.659 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:53.659 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 67, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 21176629, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:53.660 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:53.661 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:53.661 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:53.661 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:53.661 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:48:53.661 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 249, 2024-05-27T08:48:53.661 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 12845, 2024-05-27T08:48:53.661 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:53.661 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:53.662 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:53.662 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 1528, 2024-05-27T08:48:53.662 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 452, 2024-05-27T08:48:53.662 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 463, 2024-05-27T08:48:53.662 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 20843 2024-05-27T08:48:53.662 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.662 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 343, 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 911, 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 462, 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 950, 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 371, 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:53.663 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 3792, 2024-05-27T08:48:53.664 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 3792, 2024-05-27T08:48:53.664 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:53.664 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-05-27T08:48:53.664 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:53.664 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:53.664 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:53.664 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-05-27T08:48:53.664 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-05-27T08:48:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 7, 2024-05-27T08:48:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 209, 2024-05-27T08:48:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-05-27T08:48:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:53.665 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-05-27T08:48:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-05-27T08:48:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-05-27T08:48:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 17247, 2024-05-27T08:48:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-05-27T08:48:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-05-27T08:48:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 17247, 2024-05-27T08:48:53.666 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-05-27T08:48:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-05-27T08:48:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 21, 2024-05-27T08:48:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-05-27T08:48:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-05-27T08:48:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 21, 2024-05-27T08:48:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 3, 2024-05-27T08:48:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-05-27T08:48:53.667 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-05-27T08:48:53.668 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-05-27T08:48:53.668 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 29908476, 2024-05-27T08:48:53.668 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-05-27T08:48:53.668 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-05-27T08:48:53.668 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4349, 2024-05-27T08:48:53.668 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 7.206440581, 2024-05-27T08:48:53.668 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001657033 2024-05-27T08:48:53.668 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.669 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-05-27T08:48:53.669 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.669 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-05-27T08:48:53.669 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 126, 2024-05-27T08:48:53.669 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 3918, 2024-05-27T08:48:53.669 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 3792, 2024-05-27T08:48:53.669 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 26, 2024-05-27T08:48:53.669 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 501, 2024-05-27T08:48:53.670 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 475, 2024-05-27T08:48:53.670 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 167, 2024-05-27T08:48:53.670 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 5674, 2024-05-27T08:48:53.670 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 5507, 2024-05-27T08:48:53.670 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 3, 2024-05-27T08:48:53.670 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 3926, 2024-05-27T08:48:53.670 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 3923, 2024-05-27T08:48:53.670 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 64444, 2024-05-27T08:48:53.671 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223608 2024-05-27T08:48:53.671 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.671 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-05-27T08:48:53.671 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 43584, 2024-05-27T08:48:53.671 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-05-27T08:48:53.671 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 24139, 2024-05-27T08:48:53.671 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 24, 2024-05-27T08:48:53.671 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 7908, 2024-05-27T08:48:53.672 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-05-27T08:48:53.672 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3792, 2024-05-27T08:48:53.672 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 8.157947033, 2024-05-27T08:48:53.672 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002151357 2024-05-27T08:48:53.672 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.672 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-05-27T08:48:53.672 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 96, 2024-05-27T08:48:53.672 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.062507183, 2024-05-27T08:48:53.673 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000651116 2024-05-27T08:48:53.673 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.673 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:53.673 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.673 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.673 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.673 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.673 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-05-27T08:48:53.674 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.674 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.674 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.674 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.674 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-05-27T08:48:53.674 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4919, 2024-05-27T08:48:53.674 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.596063934, 2024-05-27T08:48:53.674 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000324469 2024-05-27T08:48:53.675 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.675 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:53.675 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.675 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.675 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.675 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.675 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-05-27T08:48:53.675 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.676 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.676 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.676 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.676 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-05-27T08:48:53.676 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.676 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.676 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.676 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.677 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:53.677 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.677 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.677 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.677 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.677 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:53.677 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.677 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.678 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.678 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.678 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-05-27T08:48:53.679 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.679 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.679 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.679 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.679 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-05-27T08:48:53.679 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.679 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.680 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.680 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.680 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-05-27T08:48:53.680 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.680 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.680 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.680 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.680 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.681 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.682 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-05-27T08:48:53.682 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 8676, 2024-05-27T08:48:53.682 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.543527495, 2024-05-27T08:48:53.682 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000408428 2024-05-27T08:48:53.682 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.682 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-05-27T08:48:53.682 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 316, 2024-05-27T08:48:53.683 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.688608850, 2024-05-27T08:48:53.683 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002179141 2024-05-27T08:48:53.683 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.683 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:53.683 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.683 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.683 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.683 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.684 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:53.684 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.684 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.684 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.684 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.684 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-05-27T08:48:53.684 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.684 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:53.685 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.686 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.686 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.686 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.686 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-05-27T08:48:53.686 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.686 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.686 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.687 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.687 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:53.687 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.687 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.687 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.687 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.687 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:53.687 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.688 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.688 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.688 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.688 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-05-27T08:48:53.688 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.688 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.688 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.688 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.689 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-05-27T08:48:53.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.689 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.689 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.689 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-05-27T08:48:53.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.689 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.690 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.690 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.690 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-05-27T08:48:53.690 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3792, 2024-05-27T08:48:53.690 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 8.837297522, 2024-05-27T08:48:53.690 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.002330510 2024-05-27T08:48:53.690 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.690 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:53.691 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:53.691 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:53.691 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.691 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.691 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.691 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.691 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.691 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-05-27T08:48:53.692 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 2, 2024-05-27T08:48:53.692 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 4, 2024-05-27T08:48:53.692 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-05-27T08:48:53.692 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 2, 2024-05-27T08:48:53.692 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-05-27T08:48:53.692 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 3833, 2024-05-27T08:48:53.692 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1916, 2024-05-27T08:48:53.692 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 559, 2024-05-27T08:48:53.693 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:53.693 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.693 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-05-27T08:48:53.693 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:53.693 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:53.693 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:53.693 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:53.693 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:53.694 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:53.694 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:53.694 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:53.694 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:53.694 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:53.694 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:53.694 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:53.695 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:53.695 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:53.695 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:53.695 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:53.695 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:53.695 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:53.695 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:53.695 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:53.696 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:53.696 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:53.696 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:53.696 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:53.696 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:53.696 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:53.696 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:53.696 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:53.697 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:53.697 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:53.697 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:53.697 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-05-27T08:48:53.697 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:53.697 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:53.697 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:53.697 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:53.698 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 317491, 2024-05-27T08:48:53.698 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 155, 2024-05-27T08:48:53.698 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:53.698 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:53.698 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-05-27T08:48:53.698 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-05-27T08:48:53.698 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:53.698 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 169, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-05-27T08:48:53.699 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 471521, 2024-05-27T08:48:53.700 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 2935, 2024-05-27T08:48:53.700 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:53.700 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-05-27T08:48:53.700 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:53.700 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-05-27T08:48:53.700 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.700 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 9157, 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 26472136, 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 33, 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 9157, 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 9157, 2024-05-27T08:48:53.701 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 15.989645686, 2024-05-27T08:48:53.702 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001746166 2024-05-27T08:48:53.702 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.702 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-05-27T08:48:53.702 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-05-27T08:48:53.702 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 9157, 2024-05-27T08:48:53.702 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 9521 2024-05-27T08:48:53.702 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.703 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 9157, 2024-05-27T08:48:53.703 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 284, 2024-05-27T08:48:53.703 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 8873, 2024-05-27T08:48:53.703 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-05-27T08:48:53.703 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-05-27T08:48:53.703 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-05-27T08:48:53.703 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-05-27T08:48:53.703 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 284, 2024-05-27T08:48:53.704 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 4633, 2024-05-27T08:48:53.704 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 179, 2024-05-27T08:48:53.704 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-05-27T08:48:53.704 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-05-27T08:48:53.704 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-05-27T08:48:53.704 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-05-27T08:48:53.704 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 3810, 2024-05-27T08:48:53.704 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-05-27T08:48:53.705 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-05-27T08:48:53.706 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-05-27T08:48:53.706 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:53.706 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-05-27T08:48:53.706 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:53.706 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-05-27T08:48:53.706 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-05-27T08:48:53.706 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-05-27T08:48:53.707 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-05-27T08:48:53.707 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-05-27T08:48:53.707 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-05-27T08:48:53.707 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-05-27T08:48:53.707 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-05-27T08:48:53.707 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-05-27T08:48:53.707 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-05-27T08:48:53.707 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-05-27T08:48:53.708 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 11, 2024-05-27T08:48:53.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 12, 2024-05-27T08:48:53.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 12, 2024-05-27T08:48:53.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-05-27T08:48:53.709 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-05-27T08:48:53.709 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 420, 2024-05-27T08:48:53.709 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-05-27T08:48:53.709 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 165 2024-05-27T08:48:53.709 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.710 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-05-27T08:48:53.710 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-05-27T08:48:53.710 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:53.710 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-05-27T08:48:53.710 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-05-27T08:48:53.710 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.710 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-05-27T08:48:53.710 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 216 2024-05-27T08:48:53.711 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.711 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-05-27T08:48:53.711 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:53.711 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 6, 2024-05-27T08:48:53.711 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-05-27T08:48:53.711 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 6, 2024-05-27T08:48:53.711 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 3792, 2024-05-27T08:48:53.711 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 3792, 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 111659, 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 26438470, 2024-05-27T08:48:53.712 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:53.713 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 111659, 2024-05-27T08:48:53.713 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:53.713 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:53.713 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 111659, 2024-05-27T08:48:53.713 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 26438470, 2024-05-27T08:48:53.713 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:53.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.713 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.714 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.714 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.714 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:53.714 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:53.714 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-27T08:48:53.714 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:53.714 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:53.715 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:53.715 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:53.715 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:53.715 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 9157, 2024-05-27T08:48:53.715 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 26855176, 2024-05-27T08:48:53.715 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 9157, 2024-05-27T08:48:53.715 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 26855176, 2024-05-27T08:48:53.715 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-05-27T08:48:53.716 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 9157, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 9157, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 9157, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 9157, 2024-05-27T08:48:53.717 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:53.718 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.718 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.718 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.718 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.718 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.718 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:53.718 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:53.718 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-27T08:48:53.719 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:53.719 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 3792, 2024-05-27T08:48:53.719 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 383040, 2024-05-27T08:48:53.719 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:53.719 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 3792, 2024-05-27T08:48:53.719 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:53.719 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:53.720 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 284, 2024-05-27T08:48:53.720 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 383040, 2024-05-27T08:48:53.720 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:53.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.720 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.720 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.720 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55971112a500": { 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 17247, 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 25714172, 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 17247, 2024-05-27T08:48:53.721 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:53.722 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:53.722 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4349, 2024-05-27T08:48:53.722 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 25714172, 2024-05-27T08:48:53.722 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:53.722 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:53.722 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:53.722 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:53.722 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.723 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:53.723 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-27T08:48:54.097 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:53 smithi179 ceph-mon[36481]: pgmap v329: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-05-27T08:48:54.116 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:53 smithi092 ceph-mon[36049]: pgmap v329: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-05-27T08:48:54.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:53 smithi077 ceph-mon[32007]: pgmap v329: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-05-27T08:48:54.409 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-27T08:48:54.410 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-05-27T08:48:54.410 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=343 2024-05-27T08:48:54.410 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=911 2024-05-27T08:48:54.410 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-27T08:48:54.411 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:54.411 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:54.411 DEBUG:tasks.check_counter:Getting stats from l 2024-05-27T08:48:54.411 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-27T08:48:54.646 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.c/config 2024-05-27T08:48:55.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:55 smithi179 ceph-mon[36481]: pgmap v330: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-05-27T08:48:56.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:55 smithi077 ceph-mon[32007]: pgmap v330: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-05-27T08:48:56.161 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-05-27T08:48:56.162 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-05-27T08:48:56.162 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1139, 2024-05-27T08:48:56.162 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1114, 2024-05-27T08:48:56.162 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 2119454, 2024-05-27T08:48:56.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 421906, 2024-05-27T08:48:56.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:56.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 1, 2024-05-27T08:48:56.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.150477887, 2024-05-27T08:48:56.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.043205046, 2024-05-27T08:48:56.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.070325521, 2024-05-27T08:48:56.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.039545898, 2024-05-27T08:48:56.163 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:56.164 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1113, 2024-05-27T08:48:56.164 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.056827739, 2024-05-27T08:48:56.164 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000051058 2024-05-27T08:48:56.164 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.164 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:56.164 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.165 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.165 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.165 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.165 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1834256, 2024-05-27T08:48:56.165 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 147472 2024-05-27T08:48:56.165 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.166 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-05-27T08:48:56.166 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 247, 2024-05-27T08:48:56.166 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 422, 2024-05-27T08:48:56.166 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 4806786, 2024-05-27T08:48:56.166 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 543164, 2024-05-27T08:48:56.167 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 7, 2024-05-27T08:48:56.167 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-05-27T08:48:56.167 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.033308409, 2024-05-27T08:48:56.167 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.012030663, 2024-05-27T08:48:56.167 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.010570782, 2024-05-27T08:48:56.167 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.007199586, 2024-05-27T08:48:56.168 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:56.168 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 420, 2024-05-27T08:48:56.168 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.021771163, 2024-05-27T08:48:56.168 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000051836 2024-05-27T08:48:56.168 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.168 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:56.168 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.169 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.169 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.169 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.169 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-27T08:48:56.169 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 466464 2024-05-27T08:48:56.169 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.170 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-05-27T08:48:56.170 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1460, 2024-05-27T08:48:56.170 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1462, 2024-05-27T08:48:56.170 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 21343373, 2024-05-27T08:48:56.170 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 464287, 2024-05-27T08:48:56.170 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 5, 2024-05-27T08:48:56.171 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-05-27T08:48:56.171 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.110589269, 2024-05-27T08:48:56.171 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.033319078, 2024-05-27T08:48:56.171 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.035899735, 2024-05-27T08:48:56.171 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.033899357, 2024-05-27T08:48:56.171 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-05-27T08:48:56.172 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1460, 2024-05-27T08:48:56.172 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.048434605, 2024-05-27T08:48:56.172 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000033174 2024-05-27T08:48:56.172 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.172 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-05-27T08:48:56.172 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.173 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.173 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.173 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.173 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-27T08:48:56.173 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-05-27T08:48:56.173 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.174 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-05-27T08:48:56.174 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-05-27T08:48:56.174 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-05-27T08:48:56.174 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.175 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-05-27T08:48:56.175 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-27T08:48:56.175 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-27T08:48:56.175 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4362, 2024-05-27T08:48:56.176 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.116644491, 2024-05-27T08:48:56.176 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000026741 2024-05-27T08:48:56.176 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.176 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.176 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-05-27T08:48:56.177 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-05-27T08:48:56.177 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-05-27T08:48:56.177 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.177 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.177 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.178 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.178 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.178 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-05-27T08:48:56.178 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2024-05-27T08:48:56.179 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2024-05-27T08:48:56.179 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-05-27T08:48:56.179 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.179 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.179 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.180 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.180 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-05-27T08:48:56.180 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-05-27T08:48:56.180 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-05-27T08:48:56.180 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-05-27T08:48:56.180 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2024-05-27T08:48:56.181 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-05-27T08:48:56.182 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-05-27T08:48:56.182 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 3967, 2024-05-27T08:48:56.182 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 27, 2024-05-27T08:48:56.183 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 3940, 2024-05-27T08:48:56.183 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-05-27T08:48:56.183 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 3929, 2024-05-27T08:48:56.183 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-05-27T08:48:56.183 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-05-27T08:48:56.183 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-05-27T08:48:56.183 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 4, 2024-05-27T08:48:56.183 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2024-05-27T08:48:56.184 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2024-05-27T08:48:56.184 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-05-27T08:48:56.184 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-05-27T08:48:56.184 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-05-27T08:48:56.184 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-05-27T08:48:56.184 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-05-27T08:48:56.184 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-05-27T08:48:56.184 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-05-27T08:48:56.185 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-05-27T08:48:56.185 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-05-27T08:48:56.185 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-05-27T08:48:56.185 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-05-27T08:48:56.185 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-05-27T08:48:56.185 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-05-27T08:48:56.185 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-05-27T08:48:56.186 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 0, 2024-05-27T08:48:56.186 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 0, 2024-05-27T08:48:56.186 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-05-27T08:48:56.186 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-05-27T08:48:56.186 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-05-27T08:48:56.186 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-05-27T08:48:56.186 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-05-27T08:48:56.187 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-05-27T08:48:56.187 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-05-27T08:48:56.187 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-05-27T08:48:56.187 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-05-27T08:48:56.187 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-05-27T08:48:56.187 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-05-27T08:48:56.187 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-05-27T08:48:56.187 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-05-27T08:48:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-27T08:48:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-27T08:48:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-05-27T08:48:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-05-27T08:48:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-05-27T08:48:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-05-27T08:48:56.188 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-05-27T08:48:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-05-27T08:48:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-05-27T08:48:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-05-27T08:48:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-05-27T08:48:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-05-27T08:48:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-05-27T08:48:56.189 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-05-27T08:48:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-05-27T08:48:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-05-27T08:48:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-05-27T08:48:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-05-27T08:48:56.190 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-05-27T08:48:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-05-27T08:48:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-05-27T08:48:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-05-27T08:48:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-05-27T08:48:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-05-27T08:48:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-05-27T08:48:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-05-27T08:48:56.191 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-05-27T08:48:56.192 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-05-27T08:48:56.192 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-05-27T08:48:56.192 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-05-27T08:48:56.192 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-05-27T08:48:56.192 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-05-27T08:48:56.192 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.192 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-05-27T08:48:56.192 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-05-27T08:48:56.193 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2024-05-27T08:48:56.193 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-05-27T08:48:56.193 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-05-27T08:48:56.193 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 17247, 2024-05-27T08:48:56.193 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-05-27T08:48:56.193 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-05-27T08:48:56.193 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2024-05-27T08:48:56.193 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-05-27T08:48:56.194 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-05-27T08:48:56.194 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 21, 2024-05-27T08:48:56.194 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 3, 2024-05-27T08:48:56.194 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-05-27T08:48:56.194 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-05-27T08:48:56.194 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-05-27T08:48:56.194 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 29908476, 2024-05-27T08:48:56.195 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 29907902, 2024-05-27T08:48:56.195 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-05-27T08:48:56.195 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.195 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.195 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.195 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.195 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 17247 2024-05-27T08:48:56.195 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.196 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-05-27T08:48:56.196 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 118, 2024-05-27T08:48:56.196 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 3910, 2024-05-27T08:48:56.196 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 3792, 2024-05-27T08:48:56.196 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 23, 2024-05-27T08:48:56.196 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 23, 2024-05-27T08:48:56.196 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 0, 2024-05-27T08:48:56.196 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 3967, 2024-05-27T08:48:56.197 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 3967, 2024-05-27T08:48:56.197 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 0, 2024-05-27T08:48:56.197 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-05-27T08:48:56.197 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2024-05-27T08:48:56.197 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-05-27T08:48:56.197 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 56072, 2024-05-27T08:48:56.197 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 215416 2024-05-27T08:48:56.197 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.198 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-05-27T08:48:56.198 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2024-05-27T08:48:56.198 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-05-27T08:48:56.198 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2024-05-27T08:48:56.198 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2024-05-27T08:48:56.198 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-05-27T08:48:56.198 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-05-27T08:48:56.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.199 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.199 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.199 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-05-27T08:48:56.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.199 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.200 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.200 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-05-27T08:48:56.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.200 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.200 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.200 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-05-27T08:48:56.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.201 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.201 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.201 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.201 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-05-27T08:48:56.201 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.201 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.201 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.201 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.202 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-05-27T08:48:56.202 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.202 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.202 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.202 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.202 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-05-27T08:48:56.202 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.202 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.203 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.203 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.203 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-05-27T08:48:56.203 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.203 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.203 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.203 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.203 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-05-27T08:48:56.204 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.204 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.204 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.204 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.204 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-05-27T08:48:56.204 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.204 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.204 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.205 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.205 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-05-27T08:48:56.205 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.205 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.205 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.205 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.205 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-05-27T08:48:56.206 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.206 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.206 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.206 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.207 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-05-27T08:48:56.207 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.207 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.207 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.207 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.207 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-05-27T08:48:56.207 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.207 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.208 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.208 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.208 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-05-27T08:48:56.208 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.208 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.208 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.208 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.209 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-05-27T08:48:56.209 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.209 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.209 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.209 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.209 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-05-27T08:48:56.209 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.210 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.210 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.210 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-05-27T08:48:56.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.210 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.210 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.211 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-05-27T08:48:56.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.211 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.211 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.211 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-05-27T08:48:56.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.212 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.212 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.212 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-05-27T08:48:56.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.212 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.212 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.213 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-05-27T08:48:56.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.213 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.213 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.213 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-05-27T08:48:56.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.213 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.214 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.214 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.214 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-05-27T08:48:56.214 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.214 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.214 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-05-27T08:48:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-05-27T08:48:56.215 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.216 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.216 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.216 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.216 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-05-27T08:48:56.216 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.216 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.216 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.216 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.217 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-05-27T08:48:56.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.217 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.217 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.217 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-05-27T08:48:56.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.217 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.218 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.218 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-05-27T08:48:56.218 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-05-27T08:48:56.218 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-05-27T08:48:56.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.218 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.219 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.219 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.219 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-05-27T08:48:56.219 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 2, 2024-05-27T08:48:56.219 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 4, 2024-05-27T08:48:56.219 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 2, 2024-05-27T08:48:56.219 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 2, 2024-05-27T08:48:56.219 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-05-27T08:48:56.220 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2024-05-27T08:48:56.220 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2024-05-27T08:48:56.220 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 3413, 2024-05-27T08:48:56.220 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-05-27T08:48:56.220 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.220 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-05-27T08:48:56.220 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-05-27T08:48:56.221 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-05-27T08:48:56.221 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-05-27T08:48:56.221 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-05-27T08:48:56.221 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-05-27T08:48:56.221 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-05-27T08:48:56.221 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-27T08:48:56.221 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-05-27T08:48:56.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-27T08:48:56.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-05-27T08:48:56.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-05-27T08:48:56.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-05-27T08:48:56.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-27T08:48:56.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-05-27T08:48:56.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-05-27T08:48:56.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-05-27T08:48:56.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-05-27T08:48:56.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-05-27T08:48:56.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-27T08:48:56.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-05-27T08:48:56.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-27T08:48:56.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-05-27T08:48:56.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-05-27T08:48:56.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-05-27T08:48:56.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-05-27T08:48:56.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-05-27T08:48:56.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-27T08:48:56.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-05-27T08:48:56.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-05-27T08:48:56.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-05-27T08:48:56.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-05-27T08:48:56.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-05-27T08:48:56.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-05-27T08:48:56.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-05-27T08:48:56.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-05-27T08:48:56.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-05-27T08:48:56.225 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 202866, 2024-05-27T08:48:56.225 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 161, 2024-05-27T08:48:56.225 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-05-27T08:48:56.225 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-05-27T08:48:56.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-05-27T08:48:56.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-05-27T08:48:56.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-05-27T08:48:56.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-05-27T08:48:56.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-05-27T08:48:56.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-05-27T08:48:56.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 8128, 2024-05-27T08:48:56.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 169, 2024-05-27T08:48:56.227 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-05-27T08:48:56.227 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-05-27T08:48:56.227 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-05-27T08:48:56.227 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-05-27T08:48:56.227 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 3129580, 2024-05-27T08:48:56.227 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 18390, 2024-05-27T08:48:56.227 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-05-27T08:48:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-05-27T08:48:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-05-27T08:48:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-05-27T08:48:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-05-27T08:48:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-05-27T08:48:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-05-27T08:48:56.228 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 2564, 2024-05-27T08:48:56.229 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 21, 2024-05-27T08:48:56.229 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 3, 2024-05-27T08:48:56.229 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 2564, 2024-05-27T08:48:56.229 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-05-27T08:48:56.229 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2564, 2024-05-27T08:48:56.229 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.276971743, 2024-05-27T08:48:56.230 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000498038 2024-05-27T08:48:56.230 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.230 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-05-27T08:48:56.230 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-05-27T08:48:56.230 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2564, 2024-05-27T08:48:56.230 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2565 2024-05-27T08:48:56.231 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.231 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 2564, 2024-05-27T08:48:56.231 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 2564, 2024-05-27T08:48:56.231 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2024-05-27T08:48:56.231 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-05-27T08:48:56.231 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-05-27T08:48:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 1527, 2024-05-27T08:48:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2024-05-27T08:48:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 1035, 2024-05-27T08:48:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2024-05-27T08:48:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2024-05-27T08:48:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-05-27T08:48:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-05-27T08:48:56.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-05-27T08:48:56.233 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-05-27T08:48:56.233 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2024-05-27T08:48:56.233 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-05-27T08:48:56.233 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-05-27T08:48:56.233 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-05-27T08:48:56.233 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 1, 2024-05-27T08:48:56.233 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2024-05-27T08:48:56.234 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-05-27T08:48:56.234 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-05-27T08:48:56.234 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-05-27T08:48:56.234 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-05-27T08:48:56.234 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-05-27T08:48:56.234 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-05-27T08:48:56.234 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-05-27T08:48:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-05-27T08:48:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-05-27T08:48:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-05-27T08:48:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-05-27T08:48:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-05-27T08:48:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-05-27T08:48:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-05-27T08:48:56.235 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-05-27T08:48:56.236 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-05-27T08:48:56.236 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-05-27T08:48:56.236 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-05-27T08:48:56.236 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-05-27T08:48:56.236 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-05-27T08:48:56.236 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-05-27T08:48:56.236 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-05-27T08:48:56.236 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-05-27T08:48:56.237 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-05-27T08:48:56.237 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-05-27T08:48:56.237 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-05-27T08:48:56.237 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-05-27T08:48:56.237 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-05-27T08:48:56.237 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 11, 2024-05-27T08:48:56.237 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 6, 2024-05-27T08:48:56.237 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 6, 2024-05-27T08:48:56.238 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-05-27T08:48:56.238 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-05-27T08:48:56.238 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2024-05-27T08:48:56.238 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2024-05-27T08:48:56.238 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2024-05-27T08:48:56.238 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.238 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.239 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-05-27T08:48:56.240 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-05-27T08:48:56.240 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-05-27T08:48:56.240 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-05-27T08:48:56.240 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-05-27T08:48:56.240 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-05-27T08:48:56.240 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-05-27T08:48:56.240 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-05-27T08:48:56.240 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.241 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-27T08:48:56.241 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:56.241 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-27T08:48:56.241 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:56.241 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 2842, 2024-05-27T08:48:56.241 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 27996699, 2024-05-27T08:48:56.241 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:56.241 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 2842, 2024-05-27T08:48:56.242 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:56.242 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:56.242 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2842, 2024-05-27T08:48:56.242 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 27996699, 2024-05-27T08:48:56.242 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:56.242 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.242 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.242 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.243 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.243 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.243 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-05-27T08:48:56.243 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:56.243 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-05-27T08:48:56.243 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:56.243 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:56.243 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:56.244 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:56.244 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:56.244 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 2564, 2024-05-27T08:48:56.244 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 25714177, 2024-05-27T08:48:56.244 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2564, 2024-05-27T08:48:56.244 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 25714177, 2024-05-27T08:48:56.245 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:56.245 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.245 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.245 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.245 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.245 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.245 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-05-27T08:48:56.245 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:56.246 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-05-27T08:48:56.246 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:56.246 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:56.246 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:56.246 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:56.246 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:56.246 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 2564, 2024-05-27T08:48:56.246 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 2564, 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2564, 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 2564, 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.247 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-05-27T08:48:56.248 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:56.248 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-27T08:48:56.248 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:56.248 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:56.248 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:56.248 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:56.248 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.249 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.250 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-05-27T08:48:56.250 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55bb2d4726e0": { 2024-05-27T08:48:56.250 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-05-27T08:48:56.250 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-05-27T08:48:56.250 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-05-27T08:48:56.250 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-05-27T08:48:56.250 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-05-27T08:48:56.250 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-05-27T08:48:56.251 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-05-27T08:48:56.251 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-05-27T08:48:56.251 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-05-27T08:48:56.251 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-05-27T08:48:56.251 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-05-27T08:48:56.251 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-05-27T08:48:56.251 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-05-27T08:48:56.251 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-05-27T08:48:56.252 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-05-27T08:48:56.252 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.252 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-05-27T08:48:56.252 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-05-27T08:48:56.253 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:55 smithi092 ceph-mon[36049]: pgmap v330: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-05-27T08:48:56.928 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-27T08:48:56.928 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-27T08:48:56.928 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-05-27T08:48:56.928 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-05-27T08:48:56.929 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-27T08:48:56.929 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-27T08:48:56.929 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-27T08:48:56.929 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-27T08:48:57.030 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-27T08:48:57.030 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:48:57.030 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:48:57.067 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-05-27T08:48:57.067 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-27T08:48:57.147 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-05-27T08:48:57.445 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:48:57.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:57 smithi077 ceph-mon[32007]: pgmap v331: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-05-27T08:48:58.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:57 smithi179 ceph-mon[36481]: pgmap v331: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-05-27T08:48:58.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:57 smithi092 ceph-mon[36049]: pgmap v331: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 11 op/s 2024-05-27T08:48:59.341 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 -- 172.21.15.77:0/3835936435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806c8c0 msgr2=0x7f1948108180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3835936435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806c8c0 0x7f1948108180 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f193c0099c0 tx=0x7f193c009cd0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 -- 172.21.15.77:0/3835936435 shutdown_connections 2024-05-27T08:48:59.342 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3835936435 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1948108860 0x7f1948110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3835936435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f194806bf10 0x7f194806c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.343 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3835936435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806c8c0 0x7f1948108180 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 -- 172.21.15.77:0/3835936435 >> 172.21.15.77:0/3835936435 conn(0x7f1948078490 msgr2=0x7f19480788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:59.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 -- 172.21.15.77:0/3835936435 shutdown_connections 2024-05-27T08:48:59.344 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.336+0000 7f194e9ee700 1 -- 172.21.15.77:0/3835936435 wait complete. 2024-05-27T08:48:59.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194e9ee700 1 Processor -- start 2024-05-27T08:48:59.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194e9ee700 1 -- start start 2024-05-27T08:48:59.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194e9ee700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806bf10 0x7f19481a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:59.345 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194e9ee700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f194806c8c0 0x7f19481a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194e9ee700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1948108860 0x7f194810f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194e9ee700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1948113790 con 0x7f194806bf10 2024-05-27T08:48:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194e9ee700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f1948113610 con 0x7f1948108860 2024-05-27T08:48:59.346 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194e9ee700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1948113910 con 0x7f194806c8c0 2024-05-27T08:48:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f1947fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806bf10 0x7f19481a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f1947fff700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806bf10 0x7f19481a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:36832/0 (socket says 172.21.15.77:36832) 2024-05-27T08:48:59.347 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f1947fff700 1 -- 172.21.15.77:0/3384828487 learned_addr learned my addr 172.21.15.77:0/3384828487 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:48:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f194cf8b700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1948108860 0x7f194810f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.337+0000 7f19477fe700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f194806c8c0 0x7f19481a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f1947fff700 1 -- 172.21.15.77:0/3384828487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f194806c8c0 msgr2=0x7f19481a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:59.348 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f1947fff700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f194806c8c0 0x7f19481a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f1947fff700 1 -- 172.21.15.77:0/3384828487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1948108860 msgr2=0x7f194810f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f1947fff700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1948108860 0x7f194810f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f1947fff700 1 -- 172.21.15.77:0/3384828487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194810feb0 con 0x7f194806bf10 2024-05-27T08:48:59.349 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f19477fe700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f194806c8c0 0x7f19481a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:48:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f1947fff700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806bf10 0x7f19481a4ee0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f193800d980 tx=0x7f193800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f19457fa700 1 -- 172.21.15.77:0/3384828487 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938017070 con 0x7f194806bf10 2024-05-27T08:48:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f19457fa700 1 -- 172.21.15.77:0/3384828487 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1938005020 con 0x7f194806bf10 2024-05-27T08:48:59.350 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f19457fa700 1 -- 172.21.15.77:0/3384828487 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938009b50 con 0x7f194806bf10 2024-05-27T08:48:59.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19481101a0 con 0x7f194806bf10 2024-05-27T08:48:59.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.338+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19481ace10 con 0x7f194806bf10 2024-05-27T08:48:59.351 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.339+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1948104cd0 con 0x7f194806bf10 2024-05-27T08:48:59.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.342+0000 7f19457fa700 1 -- 172.21.15.77:0/3384828487 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f19380051d0 con 0x7f194806bf10 2024-05-27T08:48:59.353 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.342+0000 7f19457fa700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f19300b29f0 0x7f19300b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:48:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.342+0000 7f19477fe700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f19300b29f0 0x7f19300b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:48:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.342+0000 7f19457fa700 1 -- 172.21.15.77:0/3384828487 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f19380e29c0 con 0x7f194806bf10 2024-05-27T08:48:59.354 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.343+0000 7f19477fe700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f19300b29f0 0x7f19300b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f193c000c00 tx=0x7f193c005fd0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:48:59.355 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.346+0000 7f19457fa700 1 -- 172.21.15.77:0/3384828487 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1938020020 con 0x7f194806bf10 2024-05-27T08:48:59.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.567+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f19481a1660 con 0x7f194806bf10 2024-05-27T08:48:59.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.568+0000 7f19457fa700 1 -- 172.21.15.77:0/3384828487 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v94) ==== 81+0+944 (secure 0 0 0) 0x7f19380054f0 con 0x7f194806bf10 2024-05-27T08:48:59.568 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3561834991 2024-05-28T08:38:16.210207+0000 2024-05-27T08:48:59.569 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/4211521509 2024-05-28T08:38:16.210207+0000 2024-05-27T08:48:59.569 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/2118880698 2024-05-28T08:38:16.210207+0000 2024-05-27T08:48:59.569 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2684021753 2024-05-28T08:27:57.064761+0000 2024-05-27T08:48:59.569 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/2456305222 2024-05-28T08:27:57.064761+0000 2024-05-27T08:48:59.569 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3434063639 2024-05-28T08:27:57.064761+0000 2024-05-27T08:48:59.569 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/2456305222 2024-05-28T08:27:57.064761+0000 2024-05-27T08:48:59.569 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6801/2203620437 2024-05-28T08:27:30.012493+0000 2024-05-27T08:48:59.569 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/21678968 2024-05-28T08:27:30.012493+0000 2024-05-27T08:48:59.570 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/2118880698 2024-05-28T08:38:16.210207+0000 2024-05-27T08:48:59.570 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2410459695 2024-05-28T08:38:16.210207+0000 2024-05-27T08:48:59.570 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3150137608 2024-05-28T08:27:57.064761+0000 2024-05-27T08:48:59.570 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:6800/2203620437 2024-05-28T08:27:30.012493+0000 2024-05-27T08:48:59.570 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/1391732673 2024-05-28T08:27:30.012493+0000 2024-05-27T08:48:59.570 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/2484667947 2024-05-28T08:38:16.210207+0000 2024-05-27T08:48:59.570 INFO:teuthology.orchestra.run.smithi077.stdout:172.21.15.77:0/3386760140 2024-05-28T08:27:30.012493+0000 2024-05-27T08:48:59.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f19300b29f0 msgr2=0x7f19300b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:59.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f19300b29f0 0x7f19300b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f193c000c00 tx=0x7f193c005fd0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.571 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806bf10 msgr2=0x7f19481a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:48:59.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806bf10 0x7f19481a4ee0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f193800d980 tx=0x7f193800dc90 comp rx=0 tx=0).stop 2024-05-27T08:48:59.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 shutdown_connections 2024-05-27T08:48:59.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f19300b29f0 0x7f19300b4eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1948108860 0x7f194810f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f194806bf10 0x7f19481a4ee0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 --2- 172.21.15.77:0/3384828487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f194806c8c0 0x7f19481a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:48:59.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 >> 172.21.15.77:0/3384828487 conn(0x7f1948078490 msgr2=0x7f1948103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:48:59.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 shutdown_connections 2024-05-27T08:48:59.573 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:48:59.570+0000 7f194e9ee700 1 -- 172.21.15.77:0/3384828487 wait complete. 2024-05-27T08:48:59.573 INFO:teuthology.orchestra.run.smithi077.stderr:listed 16 entries 2024-05-27T08:48:59.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:59 smithi077 ceph-mon[32007]: pgmap v332: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-27T08:48:59.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:48:59 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3384828487' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:49:00.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:59 smithi179 ceph-mon[36481]: pgmap v332: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-27T08:49:00.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:48:59 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3384828487' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:49:00.166 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-27T08:49:00.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:49:00.167 DEBUG:teuthology.orchestra.run.smithi077:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-27T08:49:00.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:59 smithi092 ceph-mon[36049]: pgmap v332: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-27T08:49:00.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:48:59 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3384828487' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-27T08:49:00.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:49:00.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:49:00.345 DEBUG:teuthology.orchestra.run.smithi077:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-27T08:49:00.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:49:00.388 DEBUG:teuthology.orchestra.run.smithi077:> ip netns list 2024-05-27T08:49:00.445 INFO:teuthology.orchestra.run.smithi077.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-27T08:49:00.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:49:00.446 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-27T08:49:00.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-27T08:49:00.512 DEBUG:teuthology.orchestra.run.smithi077:> sudo ip link delete ceph-brx 2024-05-27T08:49:00.620 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-27T08:49:00.637 INFO:tasks.cephadm:Teardown begin 2024-05-27T08:49:00.638 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-27T08:49:00.641 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:49:00.883 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:49:02.035 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:01 smithi077 ceph-mon[32007]: pgmap v333: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-27T08:49:02.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:01 smithi179 ceph-mon[36481]: pgmap v333: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-27T08:49:02.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:01 smithi092 ceph-mon[36049]: pgmap v333: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 10 op/s 2024-05-27T08:49:02.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/4158285877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941021d0 msgr2=0x7f5f941025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:02.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/4158285877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941021d0 0x7f5f941025b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5f90009990 tx=0x7f5f90009ca0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/4158285877 shutdown_connections 2024-05-27T08:49:02.533 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/4158285877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f9410fd50 0x7f5f941121f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/4158285877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941021d0 0x7f5f941025b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/4158285877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f94102b80 0x7f5f9410f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/4158285877 >> 172.21.15.77:0/4158285877 conn(0x7f5f940776b0 msgr2=0x7f5f94077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:02.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/4158285877 shutdown_connections 2024-05-27T08:49:02.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.495+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/4158285877 wait complete. 2024-05-27T08:49:02.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f9b2cb700 1 Processor -- start 2024-05-27T08:49:02.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f9b2cb700 1 -- start start 2024-05-27T08:49:02.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f9b2cb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f941021d0 0x7f5f9419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:02.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f9b2cb700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94102b80 0x7f5f9419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:02.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f9b2cb700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410fd50 0x7f5f941a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:02.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f9b2cb700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94115a10 con 0x7f5f9410fd50 2024-05-27T08:49:02.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f9b2cb700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94115890 con 0x7f5f94102b80 2024-05-27T08:49:02.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f9b2cb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94115b90 con 0x7f5f941021d0 2024-05-27T08:49:02.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410fd50 0x7f5f941a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:02.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410fd50 0x7f5f941a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54206/0 (socket says 172.21.15.77:54206) 2024-05-27T08:49:02.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 -- 172.21.15.77:0/1150109521 learned_addr learned my addr 172.21.15.77:0/1150109521 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f98866700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94102b80 0x7f5f9419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99067700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f941021d0 0x7f5f9419bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 -- 172.21.15.77:0/1150109521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f941021d0 msgr2=0x7f5f9419bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f941021d0 0x7f5f9419bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 -- 172.21.15.77:0/1150109521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94102b80 msgr2=0x7f5f9419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94102b80 0x7f5f9419c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 -- 172.21.15.77:0/1150109521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f941a0fa0 con 0x7f5f9410fd50 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f99868700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410fd50 0x7f5f941a0860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00b970 tx=0x7f5f8c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:02.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.496+0000 7f5f8a7fc700 1 -- 172.21.15.77:0/1150109521 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00d860 con 0x7f5f9410fd50 2024-05-27T08:49:02.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.497+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f941a1290 con 0x7f5f9410fd50 2024-05-27T08:49:02.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.497+0000 7f5f8a7fc700 1 -- 172.21.15.77:0/1150109521 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8c00e430 con 0x7f5f9410fd50 2024-05-27T08:49:02.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.497+0000 7f5f8a7fc700 1 -- 172.21.15.77:0/1150109521 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c017660 con 0x7f5f9410fd50 2024-05-27T08:49:02.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.497+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f941a8b10 con 0x7f5f9410fd50 2024-05-27T08:49:02.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.498+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9406a230 con 0x7f5f9410fd50 2024-05-27T08:49:02.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.499+0000 7f5f8a7fc700 1 -- 172.21.15.77:0/1150109521 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5f8c00da00 con 0x7f5f9410fd50 2024-05-27T08:49:02.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.499+0000 7f5f8a7fc700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5f800b29f0 0x7f5f800b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:02.537 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.500+0000 7f5f99067700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5f800b29f0 0x7f5f800b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:02.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.500+0000 7f5f8a7fc700 1 -- 172.21.15.77:0/1150109521 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f5f8c0e23f0 con 0x7f5f9410fd50 2024-05-27T08:49:02.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.500+0000 7f5f99067700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5f800b29f0 0x7f5f800b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000b5c0 tx=0x7f5f90005960 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:02.538 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.503+0000 7f5f8a7fc700 1 -- 172.21.15.77:0/1150109521 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5f8c053df0 con 0x7f5f9410fd50 2024-05-27T08:49:02.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.752+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f9402e560 con 0x7f5f9410fd50 2024-05-27T08:49:02.753 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.753+0000 7f5f8a7fc700 1 -- 172.21.15.77:0/1150109521 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11934 (secure 0 0 0) 0x7f5f8c013030 con 0x7f5f9410fd50 2024-05-27T08:49:02.754 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:49:02.754 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:48:33.678130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:49:02.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.755+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5f800b29f0 msgr2=0x7f5f800b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:02.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.755+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5f800b29f0 0x7f5f800b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000b5c0 tx=0x7f5f90005960 comp rx=0 tx=0).stop 2024-05-27T08:49:02.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.755+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410fd50 msgr2=0x7f5f941a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:02.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410fd50 0x7f5f941a0860 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00b970 tx=0x7f5f8c00bc80 comp rx=0 tx=0).stop 2024-05-27T08:49:02.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 shutdown_connections 2024-05-27T08:49:02.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f5f800b29f0 0x7f5f800b4eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410fd50 0x7f5f941a0860 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5f941021d0 0x7f5f9419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 --2- 172.21.15.77:0/1150109521 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5f94102b80 0x7f5f9419c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:02.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 >> 172.21.15.77:0/1150109521 conn(0x7f5f940776b0 msgr2=0x7f5f94110400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:02.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 shutdown_connections 2024-05-27T08:49:02.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:02.756+0000 7f5f9b2cb700 1 -- 172.21.15.77:0/1150109521 wait complete. 2024-05-27T08:49:02.758 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 41 2024-05-27T08:49:03.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:02 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1150109521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:49:03.120 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:02 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1150109521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:49:03.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:02 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1150109521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:49:03.434 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-05-27T08:49:03.677 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:49:04.040 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:03 smithi077 ceph-mon[32007]: pgmap v334: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-05-27T08:49:04.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:03 smithi179 ceph-mon[36481]: pgmap v334: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-05-27T08:49:04.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:03 smithi092 ceph-mon[36049]: pgmap v334: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-05-27T08:49:05.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/1561212469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b741084b0 msgr2=0x7f9b74108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:05.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/1561212469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b741084b0 0x7f9b74108890 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009990 tx=0x7f9b68009ca0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.276 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/1561212469 shutdown_connections 2024-05-27T08:49:05.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/1561212469 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b74102e30 0x7f9b7410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/1561212469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b741084b0 0x7f9b74108890 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/1561212469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b74102470 0x7f9b741028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/1561212469 >> 172.21.15.77:0/1561212469 conn(0x7f9b74079410 msgr2=0x7f9b74079820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:05.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/1561212469 shutdown_connections 2024-05-27T08:49:05.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.273+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/1561212469 wait complete. 2024-05-27T08:49:05.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7ae9f700 1 Processor -- start 2024-05-27T08:49:05.277 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7ae9f700 1 -- start start 2024-05-27T08:49:05.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7ae9f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b74102470 0x7f9b74104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:05.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7ae9f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b74102e30 0x7f9b741045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:05.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7ae9f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b741084b0 0x7f9b74104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:05.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7ae9f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9b74112640 con 0x7f9b74102470 2024-05-27T08:49:05.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7ae9f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f9b741124c0 con 0x7f9b741084b0 2024-05-27T08:49:05.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7ae9f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b741127c0 con 0x7f9b74102e30 2024-05-27T08:49:05.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b78c3b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b74102470 0x7f9b74104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:05.278 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b78c3b700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b74102470 0x7f9b74104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54218/0 (socket says 172.21.15.77:54218) 2024-05-27T08:49:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b78c3b700 1 -- 172.21.15.77:0/2453366238 learned_addr learned my addr 172.21.15.77:0/2453366238 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:49:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b73fff700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b74102e30 0x7f9b741045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7943c700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b741084b0 0x7f9b74104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b78c3b700 1 -- 172.21.15.77:0/2453366238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b74102e30 msgr2=0x7f9b741045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b78c3b700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b74102e30 0x7f9b741045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b78c3b700 1 -- 172.21.15.77:0/2453366238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b741084b0 msgr2=0x7f9b74104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b78c3b700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b741084b0 0x7f9b74104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.279 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b78c3b700 1 -- 172.21.15.77:0/2453366238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b741a8200 con 0x7f9b74102470 2024-05-27T08:49:05.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.274+0000 7f9b7943c700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b741084b0 0x7f9b74104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:49:05.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.275+0000 7f9b78c3b700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b74102470 0x7f9b74104080 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9b68000c00 tx=0x7f9b680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:05.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.275+0000 7f9b71ffb700 1 -- 172.21.15.77:0/2453366238 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68027070 con 0x7f9b74102470 2024-05-27T08:49:05.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.275+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b741a8490 con 0x7f9b74102470 2024-05-27T08:49:05.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.275+0000 7f9b71ffb700 1 -- 172.21.15.77:0/2453366238 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b680040e0 con 0x7f9b74102470 2024-05-27T08:49:05.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.275+0000 7f9b71ffb700 1 -- 172.21.15.77:0/2453366238 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6800f460 con 0x7f9b74102470 2024-05-27T08:49:05.280 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.275+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b741a88e0 con 0x7f9b74102470 2024-05-27T08:49:05.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.276+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7406a230 con 0x7f9b74102470 2024-05-27T08:49:05.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.277+0000 7f9b71ffb700 1 -- 172.21.15.77:0/2453366238 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9b6802a070 con 0x7f9b74102470 2024-05-27T08:49:05.281 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.278+0000 7f9b71ffb700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9b640b29f0 0x7f9b640b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:05.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.278+0000 7f9b73fff700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9b640b29f0 0x7f9b640b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:05.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.278+0000 7f9b71ffb700 1 -- 172.21.15.77:0/2453366238 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f9b680e3ab0 con 0x7f9b74102470 2024-05-27T08:49:05.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.279+0000 7f9b73fff700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9b640b29f0 0x7f9b640b4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b74079c10 tx=0x7f9b6000a300 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:05.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.282+0000 7f9b71ffb700 1 -- 172.21.15.77:0/2453366238 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b68031070 con 0x7f9b74102470 2024-05-27T08:49:05.531 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.530+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b7402e560 con 0x7f9b74102470 2024-05-27T08:49:05.532 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.531+0000 7f9b71ffb700 1 -- 172.21.15.77:0/2453366238 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11934 (secure 0 0 0) 0x7f9b68013070 con 0x7f9b74102470 2024-05-27T08:49:05.532 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:49:05.532 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14520,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3520949826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3520949826},{"type":"v1","addr":"172.21.15.77:6835","nonce":3520949826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14532,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/619760507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":619760507},{"type":"v1","addr":"172.21.15.77:6837","nonce":619760507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-27T08:35:08.767215+0000","modified":"2024-05-27T08:48:33.678130+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24433,"mds_2":24418,"mds_3":24412,"mds_4":24410},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6839/3869213740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":3869213740},{"type":"v1","addr":"172.21.15.77:6839","nonce":3869213740}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6841/906525687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6840","nonce":906525687},{"type":"v1","addr":"172.21.15.77:6841","nonce":906525687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6833/1631050328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":1631050328},{"type":"v1","addr":"172.21.15.92:6833","nonce":1631050328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6833/1008462834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":1008462834},{"type":"v1","addr":"172.21.15.179:6833","nonce":1008462834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.92:6835/4019780450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6834","nonce":4019780450},{"type":"v1","addr":"172.21.15.92:6835","nonce":4019780450}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24410":{"gid":24410,"name":"h","rank":4,"incarnation":25,"state":"up:active","state_seq":12,"addr":"172.21.15.92:6837/2920087591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":2920087591},{"type":"v1","addr":"172.21.15.92:6837","nonce":2920087591}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24412":{"gid":24412,"name":"f","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.179:6835/2178851321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2178851321},{"type":"v1","addr":"172.21.15.179:6835","nonce":2178851321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.179:6837/2857088493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":2857088493},{"type":"v1","addr":"172.21.15.179:6837","nonce":2857088493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/2672999131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2672999131},{"type":"v1","addr":"172.21.15.179:6839","nonce":2672999131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.92:6839/709031605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6838","nonce":709031605},{"type":"v1","addr":"172.21.15.92:6839","nonce":709031605}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24418,24410,24433,24412]},"id":1}]} 2024-05-27T08:49:05.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9b640b29f0 msgr2=0x7f9b640b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:05.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9b640b29f0 0x7f9b640b4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b74079c10 tx=0x7f9b6000a300 comp rx=0 tx=0).stop 2024-05-27T08:49:05.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b74102470 msgr2=0x7f9b74104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:05.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b74102470 0x7f9b74104080 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9b68000c00 tx=0x7f9b680047b0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 shutdown_connections 2024-05-27T08:49:05.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f9b640b29f0 0x7f9b640b4eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b741084b0 0x7f9b74104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9b74102470 0x7f9b74104080 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 --2- 172.21.15.77:0/2453366238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b74102e30 0x7f9b741045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:05.535 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 >> 172.21.15.77:0/2453366238 conn(0x7f9b74079410 msgr2=0x7f9b7406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:05.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 shutdown_connections 2024-05-27T08:49:05.536 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:05.533+0000 7f9b7ae9f700 1 -- 172.21.15.77:0/2453366238 wait complete. 2024-05-27T08:49:05.536 INFO:teuthology.orchestra.run.smithi077.stderr:dumped fsmap epoch 41 2024-05-27T08:49:05.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:05 smithi077 ceph-mon[32007]: pgmap v335: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:49:05.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:05 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2453366238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:49:06.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:05 smithi179 ceph-mon[36481]: pgmap v335: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:49:06.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:05 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2453366238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:49:06.120 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-05-27T08:49:06.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:05 smithi092 ceph-mon[36049]: pgmap v335: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:49:06.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:05 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2453366238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-27T08:49:06.360 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:49:08.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 -- 172.21.15.77:0/1992380767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc102e30 msgr2=0x7f7cbc10f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:08.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/1992380767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc102e30 0x7f7cbc10f3f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b0a0 tx=0x7f7cb400b3b0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.067 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 -- 172.21.15.77:0/1992380767 shutdown_connections 2024-05-27T08:49:08.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/1992380767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc102e30 0x7f7cbc10f3f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/1992380767 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cbc1042f0 0x7f7cbc1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/1992380767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cbc102470 0x7f7cbc1028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 -- 172.21.15.77:0/1992380767 >> 172.21.15.77:0/1992380767 conn(0x7f7cbc0784b0 msgr2=0x7f7cbc0788c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:08.068 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 -- 172.21.15.77:0/1992380767 shutdown_connections 2024-05-27T08:49:08.068 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:07 smithi077 ceph-mon[32007]: pgmap v336: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:49:08.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.065+0000 7f7cc2b69700 1 -- 172.21.15.77:0/1992380767 wait complete. 2024-05-27T08:49:08.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.066+0000 7f7cc2b69700 1 Processor -- start 2024-05-27T08:49:08.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.066+0000 7f7cc2b69700 1 -- start start 2024-05-27T08:49:08.070 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.066+0000 7f7cc2b69700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cbc102470 0x7f7cbc19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.066+0000 7f7cc2b69700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cbc102e30 0x7f7cbc19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.066+0000 7f7cc2b69700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc1042f0 0x7f7cbc1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.066+0000 7f7cc2b69700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc112740 con 0x7f7cbc1042f0 2024-05-27T08:49:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.066+0000 7f7cc2b69700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1125c0 con 0x7f7cbc102e30 2024-05-27T08:49:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.066+0000 7f7cc2b69700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7cbc1128c0 con 0x7f7cbc102470 2024-05-27T08:49:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc1042f0 0x7f7cbc1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc1042f0 0x7f7cbc1a06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54232/0 (socket says 172.21.15.77:54232) 2024-05-27T08:49:08.071 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 -- 172.21.15.77:0/3877814123 learned_addr learned my addr 172.21.15.77:0/3877814123 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:49:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cbbfff700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cbc102e30 0x7f7cbc19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc0905700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cbc102470 0x7f7cbc19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 -- 172.21.15.77:0/3877814123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cbc102470 msgr2=0x7f7cbc19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cbc102470 0x7f7cbc19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 -- 172.21.15.77:0/3877814123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cbc102e30 msgr2=0x7f7cbc19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:08.072 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cbc102e30 0x7f7cbc19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 -- 172.21.15.77:0/3877814123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cbc1a0e20 con 0x7f7cbc1042f0 2024-05-27T08:49:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc0905700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cbc102470 0x7f7cbc19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:49:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cbbfff700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cbc102e30 0x7f7cbc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-27T08:49:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc1106700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc1042f0 0x7f7cbc1a06e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40062a0 tx=0x7f7cb4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cb9ffb700 1 -- 172.21.15.77:0/3877814123 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4021070 con 0x7f7cbc1042f0 2024-05-27T08:49:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cb9ffb700 1 -- 172.21.15.77:0/3877814123 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb4003c00 con 0x7f7cbc1042f0 2024-05-27T08:49:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cbc1a10b0 con 0x7f7cbc1042f0 2024-05-27T08:49:08.073 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.067+0000 7f7cb9ffb700 1 -- 172.21.15.77:0/3877814123 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4004830 con 0x7f7cbc1042f0 2024-05-27T08:49:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.068+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cbc1a8990 con 0x7f7cbc1042f0 2024-05-27T08:49:08.074 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.069+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cbc06a230 con 0x7f7cbc1042f0 2024-05-27T08:49:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.070+0000 7f7cb9ffb700 1 -- 172.21.15.77:0/3877814123 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7cb4007a00 con 0x7f7cbc1042f0 2024-05-27T08:49:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.071+0000 7f7cb9ffb700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7cac0b29f0 0x7f7cac0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.071+0000 7f7cc0905700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7cac0b29f0 0x7f7cac0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.071+0000 7f7cb9ffb700 1 -- 172.21.15.77:0/3877814123 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f7cb400f020 con 0x7f7cbc1042f0 2024-05-27T08:49:08.075 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.071+0000 7f7cc0905700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7cac0b29f0 0x7f7cac0b4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009cb0 tx=0x7f7cb0009260 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:08.077 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.076+0000 7f7cb9ffb700 1 -- 172.21.15.77:0/3877814123 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cb40e5e20 con 0x7f7cbc1042f0 2024-05-27T08:49:08.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:07 smithi179 ceph-mon[36481]: pgmap v336: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:49:08.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:07 smithi092 ceph-mon[36049]: pgmap v336: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 28 op/s 2024-05-27T08:49:08.282 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.280+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cbc1a8bc0 con 0x7f7cbc1042f0 2024-05-27T08:49:08.283 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.282+0000 7f7cb9ffb700 1 -- 172.21.15.77:0/3877814123 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+23144 (secure 0 0 0) 0x7f7cb402a020 con 0x7f7cbc1042f0 2024-05-27T08:49:08.284 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-05-27T08:49:08.284 INFO:teuthology.orchestra.run.smithi077.stdout:{"epoch":94,"fsid":"c4eeaaa6-1c02-11ef-bc9b-c7b262605968","created":"2024-05-27T08:26:55.341917+0000","modified":"2024-05-27T08:48:10.631516+0000","last_up_change":"2024-05-27T08:34:06.813976+0000","last_in_change":"2024-05-27T08:33:47.676732+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-27T08:30:25.158063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-27T08:35:02.123294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-27T08:35:05.632802+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":93,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2957ea67-5b11-4557-ab6d-ec841890c830","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6803","nonce":3594583548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6805","nonce":3594583548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6809","nonce":3594583548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":3594583548},{"type":"v1","addr":"172.21.15.77:6807","nonce":3594583548}]},"public_addr":"172.21.15.77:6803/3594583548","cluster_addr":"172.21.15.77:6805/3594583548","heartbeat_back_addr":"172.21.15.77:6809/3594583548","heartbeat_front_addr":"172.21.15.77:6807/3594583548","state":["exists","up"]},{"osd":1,"uuid":"128362ac-83fc-46b5-a7aa-a7c48e49c0d2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6801","nonce":4268921047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6803","nonce":4268921047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6807","nonce":4268921047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":4268921047},{"type":"v1","addr":"172.21.15.92:6805","nonce":4268921047}]},"public_addr":"172.21.15.92:6801/4268921047","cluster_addr":"172.21.15.92:6803/4268921047","heartbeat_back_addr":"172.21.15.92:6807/4268921047","heartbeat_front_addr":"172.21.15.92:6805/4268921047","state":["exists","up"]},{"osd":2,"uuid":"8ae8a416-499c-4a05-9389-9dac4e5415de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6801","nonce":2614728525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6803","nonce":2614728525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6807","nonce":2614728525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":2614728525},{"type":"v1","addr":"172.21.15.179:6805","nonce":2614728525}]},"public_addr":"172.21.15.179:6801/2614728525","cluster_addr":"172.21.15.179:6803/2614728525","heartbeat_back_addr":"172.21.15.179:6807/2614728525","heartbeat_front_addr":"172.21.15.179:6805/2614728525","state":["exists","up"]},{"osd":3,"uuid":"96e70a03-bba7-4d18-a3f4-d14c454f719f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6811","nonce":2112674768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6813","nonce":2112674768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6817","nonce":2112674768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":2112674768},{"type":"v1","addr":"172.21.15.77:6815","nonce":2112674768}]},"public_addr":"172.21.15.77:6811/2112674768","cluster_addr":"172.21.15.77:6813/2112674768","heartbeat_back_addr":"172.21.15.77:6817/2112674768","heartbeat_front_addr":"172.21.15.77:6815/2112674768","state":["exists","up"]},{"osd":4,"uuid":"d93ff380-9258-4203-9826-6ddedfa1d03b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6809","nonce":2588881840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6811","nonce":2588881840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6815","nonce":2588881840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":2588881840},{"type":"v1","addr":"172.21.15.92:6813","nonce":2588881840}]},"public_addr":"172.21.15.92:6809/2588881840","cluster_addr":"172.21.15.92:6811/2588881840","heartbeat_back_addr":"172.21.15.92:6815/2588881840","heartbeat_front_addr":"172.21.15.92:6813/2588881840","state":["exists","up"]},{"osd":5,"uuid":"c05fe340-efa9-43dd-ac86-7d0f436df4a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6809","nonce":1761235705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6811","nonce":1761235705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6815","nonce":1761235705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":1761235705},{"type":"v1","addr":"172.21.15.179:6813","nonce":1761235705}]},"public_addr":"172.21.15.179:6809/1761235705","cluster_addr":"172.21.15.179:6811/1761235705","heartbeat_back_addr":"172.21.15.179:6815/1761235705","heartbeat_front_addr":"172.21.15.179:6813/1761235705","state":["exists","up"]},{"osd":6,"uuid":"f7d757c8-a1fa-497f-afcf-0cc1d56f7ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6819","nonce":3753560520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6821","nonce":3753560520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6825","nonce":3753560520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":3753560520},{"type":"v1","addr":"172.21.15.77:6823","nonce":3753560520}]},"public_addr":"172.21.15.77:6819/3753560520","cluster_addr":"172.21.15.77:6821/3753560520","heartbeat_back_addr":"172.21.15.77:6825/3753560520","heartbeat_front_addr":"172.21.15.77:6823/3753560520","state":["exists","up"]},{"osd":7,"uuid":"f6262ef1-767f-4b5f-a376-f6a4cb631146","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6817","nonce":3593066168}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6819","nonce":3593066168}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6823","nonce":3593066168}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":3593066168},{"type":"v1","addr":"172.21.15.92:6821","nonce":3593066168}]},"public_addr":"172.21.15.92:6817/3593066168","cluster_addr":"172.21.15.92:6819/3593066168","heartbeat_back_addr":"172.21.15.92:6823/3593066168","heartbeat_front_addr":"172.21.15.92:6821/3593066168","state":["exists","up"]},{"osd":8,"uuid":"018d4143-7dfd-40d3-9b26-758ce6acb695","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6817","nonce":3383738414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6819","nonce":3383738414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6823","nonce":3383738414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":3383738414},{"type":"v1","addr":"172.21.15.179:6821","nonce":3383738414}]},"public_addr":"172.21.15.179:6817/3383738414","cluster_addr":"172.21.15.179:6819/3383738414","heartbeat_back_addr":"172.21.15.179:6823/3383738414","heartbeat_front_addr":"172.21.15.179:6821/3383738414","state":["exists","up"]},{"osd":9,"uuid":"058309ee-c447-48af-aec5-768f8c9f5222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6827","nonce":4049231537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6829","nonce":4049231537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6833","nonce":4049231537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":4049231537},{"type":"v1","addr":"172.21.15.77:6831","nonce":4049231537}]},"public_addr":"172.21.15.77:6827/4049231537","cluster_addr":"172.21.15.77:6829/4049231537","heartbeat_back_addr":"172.21.15.77:6833/4049231537","heartbeat_front_addr":"172.21.15.77:6831/4049231537","state":["exists","up"]},{"osd":10,"uuid":"0a68bc35-c24b-4bf2-911f-2f711c0bd59a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6825","nonce":2191388192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6827","nonce":2191388192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6831","nonce":2191388192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":2191388192},{"type":"v1","addr":"172.21.15.92:6829","nonce":2191388192}]},"public_addr":"172.21.15.92:6825/2191388192","cluster_addr":"172.21.15.92:6827/2191388192","heartbeat_back_addr":"172.21.15.92:6831/2191388192","heartbeat_front_addr":"172.21.15.92:6829/2191388192","state":["exists","up"]},{"osd":11,"uuid":"1bfbc7de-1ae1-4b22-ae0e-a4bd012cef1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6825","nonce":113533236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6827","nonce":113533236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6831","nonce":113533236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":113533236},{"type":"v1","addr":"172.21.15.179:6829","nonce":113533236}]},"public_addr":"172.21.15.179:6825/113533236","cluster_addr":"172.21.15.179:6827/113533236","heartbeat_back_addr":"172.21.15.179:6831/113533236","heartbeat_front_addr":"172.21.15.179:6829/113533236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:33.098769+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:29:57.633806+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:21.952368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:30:46.476011+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:10.486814+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:31:34.938006+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:00.310524+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:25.513132+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:32:49.588366+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:15.542039+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:33:40.675479+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-27T08:34:05.543513+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.77:0/3561834991":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/4211521509":"2024-05-28T08:38:16.210207+0000","172.21.15.77:6801/2118880698":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/2684021753":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:0/3434063639":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2456305222":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6801/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/21678968":"2024-05-28T08:27:30.012493+0000","172.21.15.77:6800/2118880698":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/2410459695":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/3150137608":"2024-05-28T08:27:57.064761+0000","172.21.15.77:6800/2203620437":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/1391732673":"2024-05-28T08:27:30.012493+0000","172.21.15.77:0/2484667947":"2024-05-28T08:38:16.210207+0000","172.21.15.77:0/3386760140":"2024-05-28T08:27:30.012493+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-27T08:49:08.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7cac0b29f0 msgr2=0x7f7cac0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:08.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7cac0b29f0 0x7f7cac0b4eb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009cb0 tx=0x7f7cb0009260 comp rx=0 tx=0).stop 2024-05-27T08:49:08.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc1042f0 msgr2=0x7f7cbc1a06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:08.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc1042f0 0x7f7cbc1a06e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40062a0 tx=0x7f7cb4004d30 comp rx=0 tx=0).stop 2024-05-27T08:49:08.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 shutdown_connections 2024-05-27T08:49:08.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f7cac0b29f0 0x7f7cac0b4eb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.286 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7cbc1042f0 0x7f7cbc1a06e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cbc102470 0x7f7cbc19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 --2- 172.21.15.77:0/3877814123 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7cbc102e30 0x7f7cbc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:08.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.284+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 >> 172.21.15.77:0/3877814123 conn(0x7f7cbc0784b0 msgr2=0x7f7cbc06bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:08.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.285+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 shutdown_connections 2024-05-27T08:49:08.287 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:08.285+0000 7f7cc2b69700 1 -- 172.21.15.77:0/3877814123 wait complete. 2024-05-27T08:49:08.877 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-05-27T08:49:09.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:08 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/3877814123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:49:09.118 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:49:09.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:08 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/3877814123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:49:09.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:08 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/3877814123' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-27T08:49:09.789 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:09 smithi077 ceph-mon[32007]: pgmap v337: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-05-27T08:49:09.789 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:09 smithi077 ceph-mon[32007]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T08:49:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:09 smithi179 ceph-mon[36481]: pgmap v337: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-05-27T08:49:10.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:09 smithi179 ceph-mon[36481]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T08:49:10.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:09 smithi092 ceph-mon[36049]: pgmap v337: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 56 KiB/s wr, 33 op/s 2024-05-27T08:49:10.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:09 smithi092 ceph-mon[36049]: osdmap e95: 12 total, 12 up, 12 in 2024-05-27T08:49:10.831 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 -- 172.21.15.77:0/1695852792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406c6d0 msgr2=0x7f8994110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:10.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 --2- 172.21.15.77:0/1695852792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406c6d0 0x7f8994110400 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b0d0 tx=0x7f898c00b3e0 comp rx=0 tx=0).stop 2024-05-27T08:49:10.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 -- 172.21.15.77:0/1695852792 shutdown_connections 2024-05-27T08:49:10.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 --2- 172.21.15.77:0/1695852792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406c6d0 0x7f8994110400 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:10.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 --2- 172.21.15.77:0/1695852792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8994109630 0x7f8994109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:10.832 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 --2- 172.21.15.77:0/1695852792 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f899406bd10 0x7f899406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:10.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 -- 172.21.15.77:0/1695852792 >> 172.21.15.77:0/1695852792 conn(0x7f89940794e0 msgr2=0x7f89940798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:10.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 -- 172.21.15.77:0/1695852792 shutdown_connections 2024-05-27T08:49:10.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 -- 172.21.15.77:0/1695852792 wait complete. 2024-05-27T08:49:10.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.830+0000 7f8999798700 1 Processor -- start 2024-05-27T08:49:10.833 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8999798700 1 -- start start 2024-05-27T08:49:10.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8999798700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406bd10 0x7f899410f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:10.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8999798700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f899406c6d0 0x7f899410fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:10.835 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8999798700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8994109630 0x7f89941a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:10.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8999798700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8994113740 con 0x7f899406bd10 2024-05-27T08:49:10.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8999798700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f89941135c0 con 0x7f8994109630 2024-05-27T08:49:10.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8999798700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f89941138c0 con 0x7f899406c6d0 2024-05-27T08:49:10.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8992ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406bd10 0x7f899410f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:10.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f89927fc700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f899406c6d0 0x7f899410fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:10.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f89937fe700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8994109630 0x7f89941a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:10.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8992ffd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406bd10 0x7f899410f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37052/0 (socket says 172.21.15.77:37052) 2024-05-27T08:49:10.836 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8992ffd700 1 -- 172.21.15.77:0/11315989 learned_addr learned my addr 172.21.15.77:0/11315989 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:49:10.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8992ffd700 1 -- 172.21.15.77:0/11315989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f899406c6d0 msgr2=0x7f899410fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:10.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8992ffd700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f899406c6d0 0x7f899410fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:10.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8992ffd700 1 -- 172.21.15.77:0/11315989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8994109630 msgr2=0x7f89941a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:10.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8992ffd700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8994109630 0x7f89941a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:10.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f8992ffd700 1 -- 172.21.15.77:0/11315989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89941a5080 con 0x7f899406bd10 2024-05-27T08:49:10.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.831+0000 7f89937fe700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8994109630 0x7f89941a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:49:10.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.832+0000 7f8992ffd700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406bd10 0x7f899410f630 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f898400c9b0 tx=0x7f898400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:10.837 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.832+0000 7f897bfff700 1 -- 172.21.15.77:0/11315989 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984007b00 con 0x7f899406bd10 2024-05-27T08:49:10.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.832+0000 7f897bfff700 1 -- 172.21.15.77:0/11315989 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898400ce50 con 0x7f899406bd10 2024-05-27T08:49:10.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.832+0000 7f897bfff700 1 -- 172.21.15.77:0/11315989 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984018660 con 0x7f899406bd10 2024-05-27T08:49:10.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.832+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89941a5370 con 0x7f899406bd10 2024-05-27T08:49:10.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.832+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89941acc20 con 0x7f899406bd10 2024-05-27T08:49:10.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.833+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899406a230 con 0x7f899406bd10 2024-05-27T08:49:10.838 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.834+0000 7f897bfff700 1 -- 172.21.15.77:0/11315989 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f898400f450 con 0x7f899406bd10 2024-05-27T08:49:10.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.835+0000 7f897bfff700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f897c0b29f0 0x7f897c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:10.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.835+0000 7f89927fc700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f897c0b29f0 0x7f897c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:10.839 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.835+0000 7f897bfff700 1 -- 172.21.15.77:0/11315989 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8586+0+0 (secure 0 0 0) 0x7f89840e2860 con 0x7f899406bd10 2024-05-27T08:49:10.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.836+0000 7f89927fc700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f897c0b29f0 0x7f897c0b4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8994104440 tx=0x7f898800b410 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:10.840 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:10.839+0000 7f897bfff700 1 -- 172.21.15.77:0/11315989 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8984014030 con 0x7f899406bd10 2024-05-27T08:49:11.088 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.086+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f89941a1400 con 0x7f899406bd10 2024-05-27T08:49:11.089 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.088+0000 7f897bfff700 1 -- 172.21.15.77:0/11315989 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) ==== 161+0+0 (secure 0 0 0) 0x7f89840aa290 con 0x7f899406bd10 2024-05-27T08:49:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f897c0b29f0 msgr2=0x7f897c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f897c0b29f0 0x7f897c0b4eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8994104440 tx=0x7f898800b410 comp rx=0 tx=0).stop 2024-05-27T08:49:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406bd10 msgr2=0x7f899410f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406bd10 0x7f899410f630 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f898400c9b0 tx=0x7f898400ccc0 comp rx=0 tx=0).stop 2024-05-27T08:49:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 shutdown_connections 2024-05-27T08:49:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f897c0b29f0 0x7f897c0b4eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:11.094 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8994109630 0x7f89941a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f899406bd10 0x7f899410f630 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 --2- 172.21.15.77:0/11315989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f899406c6d0 0x7f899410fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 >> 172.21.15.77:0/11315989 conn(0x7f89940794e0 msgr2=0x7f8994100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.092+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 shutdown_connections 2024-05-27T08:49:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:11.093+0000 7f8999798700 1 -- 172.21.15.77:0/11315989 wait complete. 2024-05-27T08:49:11.095 INFO:teuthology.orchestra.run.smithi077.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-27T08:49:11.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a[32003]: 2024-05-27T08:49:11.087+0000 7fadaf082700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T08:49:11.379 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a[73999]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.379 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a[73999]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.379 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a[73999]: starting mds.a at 2024-05-27T08:49:11.380 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:49:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g[73556]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.380 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:49:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g[73556]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.380 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:49:11 smithi077 ceph-mds[73560]: -- 172.21.15.77:0/3878276741 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff32b51a40 con 0x55ff32b25000 2024-05-27T08:49:11.380 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:49:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g[73556]: starting mds.g at 2024-05-27T08:49:11.485 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h[66332]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.485 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h[66332]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.485 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mds[66336]: -- 172.21.15.92:0/2104979511 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d657b9a40 con 0x557d6578c800 2024-05-27T08:49:11.486 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h[66332]: starting mds.h at 2024-05-27T08:49:11.486 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k[65718]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.486 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k[65718]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.486 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k[65718]: starting mds.k at 2024-05-27T08:49:11.490 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.490 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.490 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mds[66653]: -- 172.21.15.92:0/2604848890 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fcf1eeda40 con 0x55fcf1ec0c00 2024-05-27T08:49:11.490 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: starting mds.b at 2024-05-27T08:49:11.491 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e[66031]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.491 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e[66031]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.491 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mds[66035]: -- 172.21.15.92:0/3971704448 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5640a3e8da40 con 0x5640a3e61000 2024-05-27T08:49:11.491 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:49:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e[66031]: starting mds.e at 2024-05-27T08:49:11.605 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f[67032]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.606 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f[67032]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.606 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mds[67036]: -- 172.21.15.179:0/3922427240 <== mon.2 v2:172.21.15.92:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d09defa40 con 0x558d09dc2c00 2024-05-27T08:49:11.606 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f[67032]: starting mds.f at 2024-05-27T08:49:11.606 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c[67331]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.606 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c[67331]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.607 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mds[67335]: -- 172.21.15.179:0/539416627 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55555bcb9a40 con 0x55555bc8cc00 2024-05-27T08:49:11.607 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c[67331]: starting mds.c at 2024-05-27T08:49:11.607 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i[67645]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.607 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i[67645]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.607 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mds[67649]: -- 172.21.15.179:0/2871126260 <== mon.0 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efd74bba40 con 0x55efd748e800 2024-05-27T08:49:11.607 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i[67645]: starting mds.i at 2024-05-27T08:49:11.608 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l[66729]: ignoring --setuser ceph since I am not root 2024-05-27T08:49:11.608 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l[66729]: ignoring --setgroup ceph since I am not root 2024-05-27T08:49:11.608 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mds[66733]: -- 172.21.15.179:0/549167848 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2fda63a40 con 0x55c2fda37000 2024-05-27T08:49:11.608 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:49:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l[66729]: starting mds.l at 2024-05-27T08:49:11.741 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-27T08:49:11.990 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:49:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mon[36481]: pgmap v339: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-05-27T08:49:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/11315989' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-27T08:49:12.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mon[36481]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-27T08:49:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mon[36481]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T08:49:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/11315989' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-27T08:49:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mon[36481]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T08:49:12.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:11 smithi179 ceph-mon[36481]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-27T08:49:12.112 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-mon[32007]: pgmap v339: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-05-27T08:49:12.113 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/11315989' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-27T08:49:12.113 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-mon[32007]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-27T08:49:12.113 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-mon[32007]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T08:49:12.113 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/11315989' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-27T08:49:12.113 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-mon[32007]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T08:49:12.113 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:11 smithi077 ceph-mon[32007]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-27T08:49:12.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mon[36049]: pgmap v339: 97 pgs: 97 active+clean; 239 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 66 KiB/s wr, 33 op/s 2024-05-27T08:49:12.235 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/11315989' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-27T08:49:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mon[36049]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-27T08:49:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mon[36049]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-27T08:49:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/11315989' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-27T08:49:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mon[36049]: osdmap e96: 12 total, 12 up, 12 in 2024-05-27T08:49:12.236 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:11 smithi092 ceph-mon[36049]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-27T08:49:13.098 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.77:6838/1429654059,v1:172.21.15.77:6839/1429654059] up:boot 2024-05-27T08:49:13.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6832/106609723,v1:172.21.15.92:6833/106609723] up:boot 2024-05-27T08:49:13.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.77:6840/500226932,v1:172.21.15.77:6841/500226932] up:boot 2024-05-27T08:49:13.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6832/2957277767,v1:172.21.15.179:6834/2957277767] up:boot 2024-05-27T08:49:13.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6833/595835866,v1:172.21.15.179:6835/595835866] up:boot 2024-05-27T08:49:13.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6834/2291462273,v1:172.21.15.92:6835/2291462273] up:boot 2024-05-27T08:49:13.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6837/2835840042,v1:172.21.15.179:6839/2835840042] up:boot 2024-05-27T08:49:13.355 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.179:6836/2563152948,v1:172.21.15.179:6838/2563152948] up:boot 2024-05-27T08:49:13.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6836/1044033809,v1:172.21.15.92:6837/1044033809] up:boot 2024-05-27T08:49:13.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: mds.? [v2:172.21.15.92:6838/3595989817,v1:172.21.15.92:6839/3595989817] up:boot 2024-05-27T08:49:13.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-27T08:49:13.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:49:13.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:49:13.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:49:13.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:49:13.356 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:49:13.357 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:49:13.357 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:49:13.357 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:49:13.357 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:49:13.357 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:49:13.357 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:13 smithi077 ceph-mon[32007]: pgmap v341: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T08:49:13.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.77:6838/1429654059,v1:172.21.15.77:6839/1429654059] up:boot 2024-05-27T08:49:13.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6832/106609723,v1:172.21.15.92:6833/106609723] up:boot 2024-05-27T08:49:13.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.77:6840/500226932,v1:172.21.15.77:6841/500226932] up:boot 2024-05-27T08:49:13.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6832/2957277767,v1:172.21.15.179:6834/2957277767] up:boot 2024-05-27T08:49:13.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6833/595835866,v1:172.21.15.179:6835/595835866] up:boot 2024-05-27T08:49:13.358 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6834/2291462273,v1:172.21.15.92:6835/2291462273] up:boot 2024-05-27T08:49:13.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6837/2835840042,v1:172.21.15.179:6839/2835840042] up:boot 2024-05-27T08:49:13.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.179:6836/2563152948,v1:172.21.15.179:6838/2563152948] up:boot 2024-05-27T08:49:13.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6836/1044033809,v1:172.21.15.92:6837/1044033809] up:boot 2024-05-27T08:49:13.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: mds.? [v2:172.21.15.92:6838/3595989817,v1:172.21.15.92:6839/3595989817] up:boot 2024-05-27T08:49:13.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-27T08:49:13.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:49:13.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:49:13.359 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:49:13.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:49:13.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:49:13.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:49:13.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:49:13.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:49:13.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:49:13.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:49:13.360 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:13 smithi179 ceph-mon[36481]: pgmap v341: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T08:49:13.415 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.77:6838/1429654059,v1:172.21.15.77:6839/1429654059] up:boot 2024-05-27T08:49:13.416 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6832/106609723,v1:172.21.15.92:6833/106609723] up:boot 2024-05-27T08:49:13.416 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.77:6840/500226932,v1:172.21.15.77:6841/500226932] up:boot 2024-05-27T08:49:13.416 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6832/2957277767,v1:172.21.15.179:6834/2957277767] up:boot 2024-05-27T08:49:13.416 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6833/595835866,v1:172.21.15.179:6835/595835866] up:boot 2024-05-27T08:49:13.416 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6834/2291462273,v1:172.21.15.92:6835/2291462273] up:boot 2024-05-27T08:49:13.416 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6837/2835840042,v1:172.21.15.179:6839/2835840042] up:boot 2024-05-27T08:49:13.416 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.179:6836/2563152948,v1:172.21.15.179:6838/2563152948] up:boot 2024-05-27T08:49:13.417 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6836/1044033809,v1:172.21.15.92:6837/1044033809] up:boot 2024-05-27T08:49:13.417 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: mds.? [v2:172.21.15.92:6838/3595989817,v1:172.21.15.92:6839/3595989817] up:boot 2024-05-27T08:49:13.417 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-27T08:49:13.417 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-27T08:49:13.417 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-27T08:49:13.417 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-27T08:49:13.417 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-27T08:49:13.417 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-27T08:49:13.418 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-27T08:49:13.418 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-27T08:49:13.418 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-27T08:49:13.418 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-27T08:49:13.418 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: from='mgr.15198 172.21.15.77:0/3441992223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-27T08:49:13.418 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:13 smithi092 ceph-mon[36049]: pgmap v341: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T08:49:13.647 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 -- 172.21.15.77:0/1605817884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801041a0 msgr2=0x7fa3801081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:13.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 --2- 172.21.15.77:0/1605817884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801041a0 0x7fa3801081c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa3700099c0 tx=0x7fa370009cd0 comp rx=0 tx=0).stop 2024-05-27T08:49:13.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 -- 172.21.15.77:0/1605817884 shutdown_connections 2024-05-27T08:49:13.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 --2- 172.21.15.77:0/1605817884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3801088f0 0x7fa380110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:13.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 --2- 172.21.15.77:0/1605817884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3801037f0 0x7fa380103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:13.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 --2- 172.21.15.77:0/1605817884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801041a0 0x7fa3801081c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:13.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 -- 172.21.15.77:0/1605817884 >> 172.21.15.77:0/1605817884 conn(0x7fa38007a6a0 msgr2=0x7fa38007aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:13.648 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 -- 172.21.15.77:0/1605817884 shutdown_connections 2024-05-27T08:49:13.649 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.647+0000 7fa3875dd700 1 -- 172.21.15.77:0/1605817884 wait complete. 2024-05-27T08:49:13.650 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa3875dd700 1 Processor -- start 2024-05-27T08:49:13.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa3875dd700 1 -- start start 2024-05-27T08:49:13.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa3875dd700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801037f0 0x7fa38019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:13.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa3875dd700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3801041a0 0x7fa38019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:13.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa3875dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3801088f0 0x7fa380199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:13.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa3875dd700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801136b0 con 0x7fa3801037f0 2024-05-27T08:49:13.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa3875dd700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fa380113530 con 0x7fa3801041a0 2024-05-27T08:49:13.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa3875dd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa380113830 con 0x7fa3801088f0 2024-05-27T08:49:13.651 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385379700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801037f0 0x7fa38019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385379700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801037f0 0x7fa38019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37112/0 (socket says 172.21.15.77:37112) 2024-05-27T08:49:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385379700 1 -- 172.21.15.77:0/2394778949 learned_addr learned my addr 172.21.15.77:0/2394778949 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:49:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa384b78700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3801041a0 0x7fa38019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385b7a700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3801088f0 0x7fa380199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385379700 1 -- 172.21.15.77:0/2394778949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3801088f0 msgr2=0x7fa380199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385379700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3801088f0 0x7fa380199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385379700 1 -- 172.21.15.77:0/2394778949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3801041a0 msgr2=0x7fa38019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:13.652 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385379700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3801041a0 0x7fa38019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa385379700 1 -- 172.21.15.77:0/2394778949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38019a090 con 0x7fa3801037f0 2024-05-27T08:49:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.648+0000 7fa384b78700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3801041a0 0x7fa38019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-27T08:49:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.649+0000 7fa385379700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801037f0 0x7fa38019f770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00d950 tx=0x7fa37c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.649+0000 7fa3767fc700 1 -- 172.21.15.77:0/2394778949 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c016070 con 0x7fa3801037f0 2024-05-27T08:49:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.649+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38019a380 con 0x7fa3801037f0 2024-05-27T08:49:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.649+0000 7fa3767fc700 1 -- 172.21.15.77:0/2394778949 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa37c00fd40 con 0x7fa3801037f0 2024-05-27T08:49:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.649+0000 7fa3767fc700 1 -- 172.21.15.77:0/2394778949 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c00e3b0 con 0x7fa3801037f0 2024-05-27T08:49:13.653 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.649+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3801a6640 con 0x7fa3801037f0 2024-05-27T08:49:13.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.650+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38006a230 con 0x7fa3801037f0 2024-05-27T08:49:13.654 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.651+0000 7fa3767fc700 1 -- 172.21.15.77:0/2394778949 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa37c019070 con 0x7fa3801037f0 2024-05-27T08:49:13.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.652+0000 7fa3767fc700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa36c0b29f0 0x7fa36c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:13.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.652+0000 7fa384b78700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa36c0b29f0 0x7fa36c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:13.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.652+0000 7fa3767fc700 1 -- 172.21.15.77:0/2394778949 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8972+0+0 (secure 0 0 0) 0x7fa37c0e22d0 con 0x7fa3801037f0 2024-05-27T08:49:13.655 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.653+0000 7fa384b78700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa36c0b29f0 0x7fa36c0b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa370009990 tx=0x7fa370000bc0 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:13.657 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.657+0000 7fa3767fc700 1 -- 172.21.15.77:0/2394778949 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa37c0a9af0 con 0x7fa3801037f0 2024-05-27T08:49:13.906 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:13.906+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa38019ae60 con 0x7fa3801037f0 2024-05-27T08:49:14.103 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.102+0000 7fa3767fc700 1 -- 172.21.15.77:0/2394778949 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) ==== 104+0+0 (secure 0 0 0) 0x7fa37c0a93f0 con 0x7fa3801037f0 2024-05-27T08:49:14.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa36c0b29f0 msgr2=0x7fa36c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:14.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa36c0b29f0 0x7fa36c0b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa370009990 tx=0x7fa370000bc0 comp rx=0 tx=0).stop 2024-05-27T08:49:14.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801037f0 msgr2=0x7fa38019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:14.107 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801037f0 0x7fa38019f770 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00d950 tx=0x7fa37c00dc60 comp rx=0 tx=0).stop 2024-05-27T08:49:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 shutdown_connections 2024-05-27T08:49:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fa36c0b29f0 0x7fa36c0b4eb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3801088f0 0x7fa380199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3801037f0 0x7fa38019f770 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 --2- 172.21.15.77:0/2394778949 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa3801041a0 0x7fa38019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 >> 172.21.15.77:0/2394778949 conn(0x7fa38007a6a0 msgr2=0x7fa380101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:14.108 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 shutdown_connections 2024-05-27T08:49:14.109 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:14.106+0000 7fa3875dd700 1 -- 172.21.15.77:0/2394778949 wait complete. 2024-05-27T08:49:14.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:14 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2394778949' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:49:14.378 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:14 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2394778949' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:49:14.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:14 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2394778949' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-27T08:49:14.730 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-27T08:49:14.970 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:49:15.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:15 smithi077 ceph-mon[32007]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-27T08:49:15.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:15 smithi077 ceph-mon[32007]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-27T08:49:15.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:15 smithi077 ceph-mon[32007]: Cluster is now healthy 2024-05-27T08:49:15.350 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:15 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2394778949' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-27T08:49:15.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:15 smithi077 ceph-mon[32007]: fsmap 12 up:standby 2024-05-27T08:49:15.351 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:15 smithi077 ceph-mon[32007]: pgmap v342: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T08:49:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:15 smithi179 ceph-mon[36481]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-27T08:49:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:15 smithi179 ceph-mon[36481]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-27T08:49:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:15 smithi179 ceph-mon[36481]: Cluster is now healthy 2024-05-27T08:49:15.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:15 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2394778949' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-27T08:49:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:15 smithi179 ceph-mon[36481]: fsmap 12 up:standby 2024-05-27T08:49:15.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:15 smithi179 ceph-mon[36481]: pgmap v342: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T08:49:15.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:15 smithi092 ceph-mon[36049]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-27T08:49:15.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:15 smithi092 ceph-mon[36049]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-27T08:49:15.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:15 smithi092 ceph-mon[36049]: Cluster is now healthy 2024-05-27T08:49:15.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:15 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2394778949' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-27T08:49:15.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:15 smithi092 ceph-mon[36049]: fsmap 12 up:standby 2024-05-27T08:49:15.486 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:15 smithi092 ceph-mon[36049]: pgmap v342: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.4 KiB/s wr, 11 op/s 2024-05-27T08:49:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 -- 172.21.15.77:0/1506018824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 msgr2=0x7f46bc102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:16.617 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 --2- 172.21.15.77:0/1506018824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 0x7f46bc102950 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009990 tx=0x7f46ac009ca0 comp rx=0 tx=0).stop 2024-05-27T08:49:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 -- 172.21.15.77:0/1506018824 shutdown_connections 2024-05-27T08:49:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 --2- 172.21.15.77:0/1506018824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46bc102e90 0x7f46bc10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 --2- 172.21.15.77:0/1506018824 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46bc1084d0 0x7f46bc1088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 --2- 172.21.15.77:0/1506018824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 0x7f46bc102950 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 -- 172.21.15.77:0/1506018824 >> 172.21.15.77:0/1506018824 conn(0x7f46bc078490 msgr2=0x7f46bc0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 -- 172.21.15.77:0/1506018824 shutdown_connections 2024-05-27T08:49:16.618 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.615+0000 7f46c450f700 1 -- 172.21.15.77:0/1506018824 wait complete. 2024-05-27T08:49:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c450f700 1 Processor -- start 2024-05-27T08:49:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c450f700 1 -- start start 2024-05-27T08:49:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c450f700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 0x7f46bc1040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c450f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46bc102e90 0x7f46bc1045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c450f700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46bc1084d0 0x7f46bc104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c450f700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc1125e0 con 0x7f46bc1024d0 2024-05-27T08:49:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c450f700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc112460 con 0x7f46bc1084d0 2024-05-27T08:49:16.619 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c450f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc112760 con 0x7f46bc102e90 2024-05-27T08:49:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c22ab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 0x7f46bc1040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c22ab700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 0x7f46bc1040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:37134/0 (socket says 172.21.15.77:37134) 2024-05-27T08:49:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.616+0000 7f46c22ab700 1 -- 172.21.15.77:0/2591310778 learned_addr learned my addr 172.21.15.77:0/2591310778 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:49:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c2aac700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46bc1084d0 0x7f46bc104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c1aaa700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46bc102e90 0x7f46bc1045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c22ab700 1 -- 172.21.15.77:0/2591310778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46bc102e90 msgr2=0x7f46bc1045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c22ab700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46bc102e90 0x7f46bc1045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:16.620 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c22ab700 1 -- 172.21.15.77:0/2591310778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46bc1084d0 msgr2=0x7f46bc104b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c22ab700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46bc1084d0 0x7f46bc104b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c22ab700 1 -- 172.21.15.77:0/2591310778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc1a80a0 con 0x7f46bc1024d0 2024-05-27T08:49:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c22ab700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 0x7f46bc1040a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f46b800d950 tx=0x7f46b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46b37fe700 1 -- 172.21.15.77:0/2591310778 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8016070 con 0x7f46bc1024d0 2024-05-27T08:49:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46b37fe700 1 -- 172.21.15.77:0/2591310778 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46b8009df0 con 0x7f46bc1024d0 2024-05-27T08:49:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc1a8390 con 0x7f46bc1024d0 2024-05-27T08:49:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46b37fe700 1 -- 172.21.15.77:0/2591310778 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b800f3c0 con 0x7f46bc1024d0 2024-05-27T08:49:16.621 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.617+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc1a88d0 con 0x7f46bc1024d0 2024-05-27T08:49:16.622 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.618+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46bc06a230 con 0x7f46bc1024d0 2024-05-27T08:49:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.619+0000 7f46b37fe700 1 -- 172.21.15.77:0/2591310778 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f46b8019070 con 0x7f46bc1024d0 2024-05-27T08:49:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.620+0000 7f46b37fe700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f46a80b29f0 0x7f46a80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.620+0000 7f46c1aaa700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f46a80b29f0 0x7f46a80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.620+0000 7f46b37fe700 1 -- 172.21.15.77:0/2591310778 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8972+0+0 (secure 0 0 0) 0x7f46b80e22a0 con 0x7f46bc1024d0 2024-05-27T08:49:16.623 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.621+0000 7f46c1aaa700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f46a80b29f0 0x7f46a80b4eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f46ac006010 tx=0x7f46ac01a040 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:16.624 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.624+0000 7f46b37fe700 1 -- 172.21.15.77:0/2591310778 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46b80a9b60 con 0x7f46bc1024d0 2024-05-27T08:49:16.829 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:16.829+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46bc1055d0 con 0x7f46bc1024d0 2024-05-27T08:49:16.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:16 smithi092 ceph-mon[36049]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T08:49:17.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:16 smithi179 ceph-mon[36481]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T08:49:17.128 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:16 smithi077 ceph-mon[32007]: osdmap e97: 12 total, 12 up, 12 in 2024-05-27T08:49:17.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.663+0000 7f46b37fe700 1 -- 172.21.15.77:0/2591310778 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7f46b80a9460 con 0x7f46bc1024d0 2024-05-27T08:49:17.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f46a80b29f0 msgr2=0x7f46a80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:17.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f46a80b29f0 0x7f46a80b4eb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f46ac006010 tx=0x7f46ac01a040 comp rx=0 tx=0).stop 2024-05-27T08:49:17.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 msgr2=0x7f46bc1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:17.667 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 0x7f46bc1040a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f46b800d950 tx=0x7f46b800dc60 comp rx=0 tx=0).stop 2024-05-27T08:49:17.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 shutdown_connections 2024-05-27T08:49:17.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7f46a80b29f0 0x7f46a80b4eb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:17.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f46bc1084d0 0x7f46bc104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:17.668 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46bc1024d0 0x7f46bc1040a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:17.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 --2- 172.21.15.77:0/2591310778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46bc102e90 0x7f46bc1045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:17.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 >> 172.21.15.77:0/2591310778 conn(0x7f46bc078490 msgr2=0x7f46bc1062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:17.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 shutdown_connections 2024-05-27T08:49:17.669 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:17.667+0000 7f46c450f700 1 -- 172.21.15.77:0/2591310778 wait complete. 2024-05-27T08:49:17.669 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_metadata' removed 2024-05-27T08:49:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:17 smithi092 ceph-mon[36049]: pgmap v344: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-27T08:49:17.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:17 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2591310778' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T08:49:17.986 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:17 smithi077 ceph-mon[32007]: pgmap v344: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-27T08:49:17.986 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:17 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2591310778' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T08:49:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:17 smithi179 ceph-mon[36481]: pgmap v344: 97 pgs: 97 active+clean; 238 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-05-27T08:49:18.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:17 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2591310778' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T08:49:18.323 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9 shell --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-27T08:49:18.582 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/mon.a/config 2024-05-27T08:49:18.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:18 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/2591310778' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-27T08:49:18.940 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:18 smithi077 ceph-mon[32007]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T08:49:18.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:18 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/2591310778' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-27T08:49:18.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:18 smithi092 ceph-mon[36049]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T08:49:19.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:18 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/2591310778' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-27T08:49:19.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:18 smithi179 ceph-mon[36481]: osdmap e98: 12 total, 12 up, 12 in 2024-05-27T08:49:19.952 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:19 smithi077 ceph-mon[32007]: pgmap v346: 65 pgs: 65 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:49:19.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:19 smithi092 ceph-mon[36049]: pgmap v346: 65 pgs: 65 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:49:20.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:19 smithi179 ceph-mon[36481]: pgmap v346: 65 pgs: 65 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:49:20.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 -- 172.21.15.77:0/3053190100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7406bab0 msgr2=0x7fba7406bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:20.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 --2- 172.21.15.77:0/3053190100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7406bab0 0x7fba7406bf30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009990 tx=0x7fba5c009ca0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 -- 172.21.15.77:0/3053190100 shutdown_connections 2024-05-27T08:49:20.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 --2- 172.21.15.77:0/3053190100 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba7406c470 0x7fba741103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 --2- 172.21.15.77:0/3053190100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba741095a0 0x7fba74109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.323 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 --2- 172.21.15.77:0/3053190100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7406bab0 0x7fba7406bf30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 -- 172.21.15.77:0/3053190100 >> 172.21.15.77:0/3053190100 conn(0x7fba740793b0 msgr2=0x7fba740797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:20.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 -- 172.21.15.77:0/3053190100 shutdown_connections 2024-05-27T08:49:20.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 -- 172.21.15.77:0/3053190100 wait complete. 2024-05-27T08:49:20.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 Processor -- start 2024-05-27T08:49:20.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 -- start start 2024-05-27T08:49:20.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba7406bab0 0x7fba74102500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:20.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba7406c470 0x7fba74102a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:20.324 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.321+0000 7fba79453700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba741095a0 0x7fba74102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:20.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba79453700 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba74113730 con 0x7fba741095a0 2024-05-27T08:49:20.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba79453700 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 -- 0x7fba741135b0 con 0x7fba7406c470 2024-05-27T08:49:20.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba79453700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fba741138b0 con 0x7fba7406bab0 2024-05-27T08:49:20.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba741095a0 0x7fba74102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:20.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba741095a0 0x7fba74102f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:40368/0 (socket says 172.21.15.77:40368) 2024-05-27T08:49:20.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba6bfff700 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba7406c470 0x7fba74102a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:20.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 -- 172.21.15.77:0/1502280320 learned_addr learned my addr 172.21.15.77:0/1502280320 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-05-27T08:49:20.325 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba72ffd700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba7406bab0 0x7fba74102500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:20.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 -- 172.21.15.77:0/1502280320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba7406bab0 msgr2=0x7fba74102500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:20.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba7406bab0 0x7fba74102500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 -- 172.21.15.77:0/1502280320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba7406c470 msgr2=0x7fba74102a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:20.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba7406c470 0x7fba74102a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 -- 172.21.15.77:0/1502280320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba741a8190 con 0x7fba741095a0 2024-05-27T08:49:20.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba737fe700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba741095a0 0x7fba74102f80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fba6c007ee0 tx=0x7fba6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:20.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba70ff9700 1 -- 172.21.15.77:0/1502280320 <== mon.0 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00dbd0 con 0x7fba741095a0 2024-05-27T08:49:20.326 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba741a8480 con 0x7fba741095a0 2024-05-27T08:49:20.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba70ff9700 1 -- 172.21.15.77:0/1502280320 <== mon.0 v2:172.21.15.77:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba6c00dd70 con 0x7fba741095a0 2024-05-27T08:49:20.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba70ff9700 1 -- 172.21.15.77:0/1502280320 <== mon.0 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c005990 con 0x7fba741095a0 2024-05-27T08:49:20.327 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.322+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba741a8990 con 0x7fba741095a0 2024-05-27T08:49:20.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.323+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7410ba80 con 0x7fba741095a0 2024-05-27T08:49:20.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.325+0000 7fba70ff9700 1 -- 172.21.15.77:0/1502280320 <== mon.0 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fba6c01a020 con 0x7fba741095a0 2024-05-27T08:49:20.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.326+0000 7fba70ff9700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fba600b29f0 0x7fba600b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-27T08:49:20.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.326+0000 7fba72ffd700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fba600b29f0 0x7fba600b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-27T08:49:20.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.326+0000 7fba70ff9700 1 -- 172.21.15.77:0/1502280320 <== mon.0 v2:172.21.15.77:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8374+0+0 (secure 0 0 0) 0x7fba6c0e23e0 con 0x7fba741095a0 2024-05-27T08:49:20.328 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.326+0000 7fba72ffd700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fba600b29f0 0x7fba600b4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba64006fd0 tx=0x7fba64008040 comp rx=0 tx=0).ready entity=mgr.15198 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-27T08:49:20.329 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.329+0000 7fba70ff9700 1 -- 172.21.15.77:0/1502280320 <== mon.0 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba6c010070 con 0x7fba741095a0 2024-05-27T08:49:20.534 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.533+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fba7410b460 con 0x7fba741095a0 2024-05-27T08:49:20.682 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.681+0000 7fba70ff9700 1 -- 172.21.15.77:0/1502280320 <== mon.0 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7fba6c0a9e50 con 0x7fba741095a0 2024-05-27T08:49:20.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fba600b29f0 msgr2=0x7fba600b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:20.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fba600b29f0 0x7fba600b4eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba64006fd0 tx=0x7fba64008040 comp rx=0 tx=0).stop 2024-05-27T08:49:20.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba741095a0 msgr2=0x7fba74102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-27T08:49:20.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba741095a0 0x7fba74102f80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fba6c007ee0 tx=0x7fba6c00d3b0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.687 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 shutdown_connections 2024-05-27T08:49:20.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:6800/4086463008,v1:172.21.15.77:6801/4086463008] conn(0x7fba600b29f0 0x7fba600b4eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba741095a0 0x7fba74102f80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba7406bab0 0x7fba74102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 --2- 172.21.15.77:0/1502280320 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fba7406c470 0x7fba74102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-27T08:49:20.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 >> 172.21.15.77:0/1502280320 conn(0x7fba740793b0 msgr2=0x7fba74101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-27T08:49:20.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 shutdown_connections 2024-05-27T08:49:20.688 INFO:teuthology.orchestra.run.smithi077.stderr:2024-05-27T08:49:20.685+0000 7fba79453700 1 -- 172.21.15.77:0/1502280320 wait complete. 2024-05-27T08:49:20.688 INFO:teuthology.orchestra.run.smithi077.stderr:pool 'cephfs_data' removed 2024-05-27T08:49:20.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:20 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1502280320' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T08:49:20.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:20 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1502280320' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T08:49:21.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:20 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1502280320' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-27T08:49:21.329 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:49:21.356 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:49:21.389 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:49:21.421 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-27T08:49:21.421 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:49:21.437 DEBUG:teuthology.orchestra.run.smithi092:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:49:21.453 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-27T08:49:21.477 INFO:tasks.cephadm:Stopping all daemons... 2024-05-27T08:49:21.477 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-27T08:49:21.478 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.a 2024-05-27T08:49:21.609 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:21 smithi077 systemd[1]: Stopping Ceph mon.a for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:21.878 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:21 smithi077 ceph-mon[32007]: pgmap v347: 65 pgs: 65 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:49:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:21 smithi077 ceph-mon[32007]: from='client.? 172.21.15.77:0/1502280320' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-27T08:49:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:21 smithi077 ceph-mon[32007]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T08:49:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:21 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a[32003]: 2024-05-27T08:49:21.806+0000 7fadb488d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:49:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:21 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a[32003]: 2024-05-27T08:49:21.806+0000 7fadb488d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-27T08:49:21.879 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:21 smithi077 podman[103004]: 2024-05-27 08:49:21.837466782 +0000 UTC m=+0.162046912 container died 59b893a782958c7fffb0ecc822012fdda7d5555885633545af796e8402489e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:49:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:21 smithi092 ceph-mon[36049]: pgmap v347: 65 pgs: 65 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:49:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:21 smithi092 ceph-mon[36049]: from='client.? 172.21.15.77:0/1502280320' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-27T08:49:21.985 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:21 smithi092 ceph-mon[36049]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T08:49:22.105 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:21 smithi179 ceph-mon[36481]: pgmap v347: 65 pgs: 65 active+clean; 29 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-27T08:49:22.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:21 smithi179 ceph-mon[36481]: from='client.? 172.21.15.77:0/1502280320' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-27T08:49:22.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:21 smithi179 ceph-mon[36481]: osdmap e99: 12 total, 12 up, 12 in 2024-05-27T08:49:22.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:22 smithi077 podman[103004]: 2024-05-27 08:49:22.197877063 +0000 UTC m=+0.522457192 container cleanup 59b893a782958c7fffb0ecc822012fdda7d5555885633545af796e8402489e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:49:22.379 INFO:journalctl@ceph.mon.a.smithi077.stdout:May 27 08:49:22 smithi077 bash[103004]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-a 2024-05-27T08:49:22.685 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.a.service' 2024-05-27T08:49:22.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:22.735 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-27T08:49:22.736 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-27T08:49:22.736 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.b 2024-05-27T08:49:23.061 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:22 smithi092 systemd[1]: Stopping Ceph mon.b for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:23.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-b[36045]: 2024-05-27T08:49:23.061+0000 7ff415669700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:49:23.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-b[36045]: 2024-05-27T08:49:23.061+0000 7ff415669700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-27T08:49:23.485 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:23 smithi092 podman[73162]: 2024-05-27 08:49:23.265286234 +0000 UTC m=+0.331994309 container died 7d3aeeafd63cd641285060f67660da6c2ecef7ffd274a641ffc9c13521bba940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T08:49:23.833 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:23 smithi092 podman[73162]: 2024-05-27 08:49:23.627620281 +0000 UTC m=+0.694328342 container cleanup 7d3aeeafd63cd641285060f67660da6c2ecef7ffd274a641ffc9c13521bba940 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-b, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-05-27T08:49:23.833 INFO:journalctl@ceph.mon.b.smithi092.stdout:May 27 08:49:23 smithi092 bash[73162]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-b 2024-05-27T08:49:24.047 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.b.service' 2024-05-27T08:49:24.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:24.100 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-27T08:49:24.101 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-27T08:49:24.101 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.c 2024-05-27T08:49:24.382 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:24 smithi179 systemd[1]: Stopping Ceph mon.c for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:24.833 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:24 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-c[36469]: 2024-05-27T08:49:24.381+0000 7fe77e778700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:49:24.833 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:24 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-c[36469]: 2024-05-27T08:49:24.381+0000 7fe77e778700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-27T08:49:24.834 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:24 smithi179 podman[73662]: 2024-05-27 08:49:24.535424143 +0000 UTC m=+0.241254376 container died 04b807590ee240fed490315e497a318d58b19430543bacdb1e9a22c066855f24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-27T08:49:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:24 smithi179 podman[73662]: 2024-05-27 08:49:24.93244006 +0000 UTC m=+0.638270294 container cleanup 04b807590ee240fed490315e497a318d58b19430543bacdb1e9a22c066855f24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T08:49:25.106 INFO:journalctl@ceph.mon.c.smithi179.stdout:May 27 08:49:24 smithi179 bash[73662]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mon-c 2024-05-27T08:49:25.366 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mon.c.service' 2024-05-27T08:49:25.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:25.412 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-27T08:49:25.412 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-27T08:49:25.412 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.x 2024-05-27T08:49:25.766 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:49:25 smithi077 systemd[1]: Stopping Ceph mgr.x for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:26.102 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:49:25 smithi077 podman[103134]: 2024-05-27 08:49:25.767571192 +0000 UTC m=+0.157978863 container died 2deab0fd4f35fb2f25a40a25d98bafcb547d7c1a77bd75ee0553cb5acb00309f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:49:26.103 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:49:26 smithi077 podman[103134]: 2024-05-27 08:49:26.103145743 +0000 UTC m=+0.493553404 container cleanup 2deab0fd4f35fb2f25a40a25d98bafcb547d7c1a77bd75ee0553cb5acb00309f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-27T08:49:26.378 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:49:26 smithi077 bash[103134]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x 2024-05-27T08:49:26.379 INFO:journalctl@ceph.mgr.x.smithi077.stdout:May 27 08:49:26 smithi077 podman[103148]: 2024-05-27 08:49:26.296219496 +0000 UTC m=+0.524156251 container remove 2deab0fd4f35fb2f25a40a25d98bafcb547d7c1a77bd75ee0553cb5acb00309f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a9cb3a581a309a99b72997ae5ddb88084f5484c9, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521) 2024-05-27T08:49:26.541 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.x.service' 2024-05-27T08:49:26.579 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:26.579 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-27T08:49:26.579 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-27T08:49:26.580 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.y 2024-05-27T08:49:26.902 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:49:26 smithi092 systemd[1]: Stopping Ceph mgr.y for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:27.182 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:49:26 smithi092 podman[73283]: 2024-05-27 08:49:26.902504818 +0000 UTC m=+0.128422928 container died 25224e5b15df5041afbbeb09e18cd233dd927c3f7bda2887b0b9d8e1513cea49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-27T08:49:27.485 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:49:27 smithi092 podman[73283]: 2024-05-27 08:49:27.281748531 +0000 UTC m=+0.507666641 container cleanup 25224e5b15df5041afbbeb09e18cd233dd927c3f7bda2887b0b9d8e1513cea49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-27T08:49:27.485 INFO:journalctl@ceph.mgr.y.smithi092.stdout:May 27 08:49:27 smithi092 bash[73283]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-y 2024-05-27T08:49:27.653 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.y.service' 2024-05-27T08:49:27.687 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:27.687 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-27T08:49:27.687 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-27T08:49:27.688 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.z 2024-05-27T08:49:28.043 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:49:27 smithi179 systemd[1]: Stopping Ceph mgr.z for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:28.321 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:49:28 smithi179 podman[73784]: 2024-05-27 08:49:28.043342865 +0000 UTC m=+0.165036853 container died 2d5719555a423277b8ed7532b7352ab40fc644562b2a76828214a4b10ff0d0bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=) 2024-05-27T08:49:28.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:49:28 smithi179 podman[73784]: 2024-05-27 08:49:28.395978369 +0000 UTC m=+0.517672348 container cleanup 2d5719555a423277b8ed7532b7352ab40fc644562b2a76828214a4b10ff0d0bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:49:28.605 INFO:journalctl@ceph.mgr.z.smithi179.stdout:May 27 08:49:28 smithi179 bash[73784]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mgr-z 2024-05-27T08:49:28.773 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mgr.z.service' 2024-05-27T08:49:28.807 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:28.807 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-27T08:49:28.807 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-27T08:49:28.807 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.0 2024-05-27T08:49:29.115 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:28 smithi077 systemd[1]: Stopping Ceph osd.0 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:29.379 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0[46521]: 2024-05-27T08:49:29.115+0000 7f2ea7f0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:49:29.379 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0[46521]: 2024-05-27T08:49:29.115+0000 7f2ea7f0d700 -1 osd.0 99 *** Got signal Terminated *** 2024-05-27T08:49:29.379 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0[46521]: 2024-05-27T08:49:29.115+0000 7f2ea7f0d700 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:49:34.965 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:34 smithi077 podman[103254]: 2024-05-27 08:49:34.676010768 +0000 UTC m=+5.673620237 container died 40a8730d1f8f468cc54872bdd64ac95942edb3b7281da1f1d0cbf60d86b077c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:49:35.378 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:34 smithi077 podman[103254]: 2024-05-27 08:49:34.965543747 +0000 UTC m=+5.963153210 container cleanup 40a8730d1f8f468cc54872bdd64ac95942edb3b7281da1f1d0cbf60d86b077c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-27T08:49:35.379 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:34 smithi077 bash[103254]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0 2024-05-27T08:49:35.379 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:35 smithi077 podman[103271]: 2024-05-27 08:49:35.191943611 +0000 UTC m=+0.514789259 container remove 40a8730d1f8f468cc54872bdd64ac95942edb3b7281da1f1d0cbf60d86b077c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-27T08:49:35.782 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:35 smithi077 podman[103358]: 2024-05-27 08:49:35.432742297 +0000 UTC m=+0.017246222 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:49:35.782 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:35 smithi077 podman[103358]: 2024-05-27 08:49:35.559390826 +0000 UTC m=+0.143894725 container create f2345573b0cd68599f023c3d88e947b8ed897fb28d149e812ac4cb9525ee6291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T08:49:35.783 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:35 smithi077 podman[103358]: 2024-05-27 08:49:35.778741805 +0000 UTC m=+0.363245719 container init f2345573b0cd68599f023c3d88e947b8ed897fb28d149e812ac4cb9525ee6291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_CLEAN=True) 2024-05-27T08:49:36.084 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:35 smithi077 podman[103358]: 2024-05-27 08:49:35.782509843 +0000 UTC m=+0.367013746 container start f2345573b0cd68599f023c3d88e947b8ed897fb28d149e812ac4cb9525ee6291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:49:36.084 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:35 smithi077 podman[103358]: 2024-05-27 08:49:35.887109629 +0000 UTC m=+0.471613564 container attach f2345573b0cd68599f023c3d88e947b8ed897fb28d149e812ac4cb9525ee6291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-27T08:49:36.362 INFO:journalctl@ceph.osd.0.smithi077.stdout:May 27 08:49:36 smithi077 podman[103358]: 2024-05-27 08:49:36.085381445 +0000 UTC m=+0.669885346 container died f2345573b0cd68599f023c3d88e947b8ed897fb28d149e812ac4cb9525ee6291 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=) 2024-05-27T08:49:36.755 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.0.service' 2024-05-27T08:49:36.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:36.795 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-27T08:49:36.795 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-27T08:49:36.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.1 2024-05-27T08:49:37.093 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:36 smithi092 systemd[1]: Stopping Ceph osd.1 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:37.485 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1[40626]: 2024-05-27T08:49:37.092+0000 7f4ea2c4f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:49:37.485 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1[40626]: 2024-05-27T08:49:37.092+0000 7f4ea2c4f700 -1 osd.1 99 *** Got signal Terminated *** 2024-05-27T08:49:37.486 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1[40626]: 2024-05-27T08:49:37.092+0000 7f4ea2c4f700 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:49:42.885 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:42 smithi092 podman[73410]: 2024-05-27 08:49:42.562903338 +0000 UTC m=+5.582592350 container died 8ba154e834df3553196534702e1fc6e4f8dc6cd0857886c9bbaf02a889c43f0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-27T08:49:43.235 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:42 smithi092 podman[73410]: 2024-05-27 08:49:42.885021144 +0000 UTC m=+5.904710153 container cleanup 8ba154e834df3553196534702e1fc6e4f8dc6cd0857886c9bbaf02a889c43f0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5) 2024-05-27T08:49:43.235 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:42 smithi092 bash[73410]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1 2024-05-27T08:49:43.235 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:43 smithi092 podman[73428]: 2024-05-27 08:49:43.103259637 +0000 UTC m=+0.530428974 container remove 8ba154e834df3553196534702e1fc6e4f8dc6cd0857886c9bbaf02a889c43f0d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T08:49:43.774 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:43 smithi092 podman[73518]: 2024-05-27 08:49:43.415887786 +0000 UTC m=+0.016957801 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:49:43.774 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:43 smithi092 podman[73518]: 2024-05-27 08:49:43.54655198 +0000 UTC m=+0.147621980 container create 4fffe22ad12828b3a22826a5e9e056fff8e475246eda902a91066099fd723c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True) 2024-05-27T08:49:43.774 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:43 smithi092 podman[73518]: 2024-05-27 08:49:43.774224247 +0000 UTC m=+0.375294255 container init 4fffe22ad12828b3a22826a5e9e056fff8e475246eda902a91066099fd723c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T08:49:44.067 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:43 smithi092 podman[73518]: 2024-05-27 08:49:43.777547739 +0000 UTC m=+0.378617755 container start 4fffe22ad12828b3a22826a5e9e056fff8e475246eda902a91066099fd723c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-27T08:49:44.067 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:43 smithi092 podman[73518]: 2024-05-27 08:49:43.874281715 +0000 UTC m=+0.475351734 container attach 4fffe22ad12828b3a22826a5e9e056fff8e475246eda902a91066099fd723c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5) 2024-05-27T08:49:44.336 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:44 smithi092 podman[73518]: 2024-05-27 08:49:44.068209228 +0000 UTC m=+0.669279244 container died 4fffe22ad12828b3a22826a5e9e056fff8e475246eda902a91066099fd723c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521) 2024-05-27T08:49:44.589 INFO:journalctl@ceph.osd.1.smithi092.stdout:May 27 08:49:44 smithi092 podman[73544]: 2024-05-27 08:49:44.586583453 +0000 UTC m=+0.513124829 container remove 4fffe22ad12828b3a22826a5e9e056fff8e475246eda902a91066099fd723c12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-27T08:49:44.609 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.1.service' 2024-05-27T08:49:44.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:44.646 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-27T08:49:44.646 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-27T08:49:44.646 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.2 2024-05-27T08:49:44.933 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:44 smithi179 systemd[1]: Stopping Ceph osd.2 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:45.203 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2[41612]: 2024-05-27T08:49:44.932+0000 7fb6e5ba9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:49:45.203 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2[41612]: 2024-05-27T08:49:44.932+0000 7fb6e5ba9700 -1 osd.2 99 *** Got signal Terminated *** 2024-05-27T08:49:45.203 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2[41612]: 2024-05-27T08:49:44.932+0000 7fb6e5ba9700 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:49:50.587 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:50 smithi179 podman[73909]: 2024-05-27 08:49:50.294381913 +0000 UTC m=+5.457764437 container died 55bb1ca76dd1090809bb2d99c39a1c81269f488f25b07029170be68e949d4708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-27T08:49:50.855 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:50 smithi179 podman[73909]: 2024-05-27 08:49:50.691161145 +0000 UTC m=+5.854543676 container cleanup 55bb1ca76dd1090809bb2d99c39a1c81269f488f25b07029170be68e949d4708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521) 2024-05-27T08:49:50.856 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:50 smithi179 bash[73909]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2 2024-05-27T08:49:51.240 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:50 smithi179 podman[73938]: 2024-05-27 08:49:50.925794917 +0000 UTC m=+0.624397228 container remove 55bb1ca76dd1090809bb2d99c39a1c81269f488f25b07029170be68e949d4708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-27T08:49:51.496 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:51 smithi179 podman[74027]: 2024-05-27 08:49:51.143035824 +0000 UTC m=+0.017249733 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:49:51.496 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:51 smithi179 podman[74027]: 2024-05-27 08:49:51.268601002 +0000 UTC m=+0.142814903 container create eb05b43e7c58e91fd2e6a0f7c3edd2a2a22faf5c73607a84ec96297835e90330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-27T08:49:51.793 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:51 smithi179 podman[74027]: 2024-05-27 08:49:51.496069164 +0000 UTC m=+0.370283074 container init eb05b43e7c58e91fd2e6a0f7c3edd2a2a22faf5c73607a84ec96297835e90330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-27T08:49:51.793 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:51 smithi179 podman[74027]: 2024-05-27 08:49:51.500124952 +0000 UTC m=+0.374338868 container start eb05b43e7c58e91fd2e6a0f7c3edd2a2a22faf5c73607a84ec96297835e90330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240521, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2) 2024-05-27T08:49:51.793 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:51 smithi179 podman[74027]: 2024-05-27 08:49:51.58758117 +0000 UTC m=+0.461795089 container attach eb05b43e7c58e91fd2e6a0f7c3edd2a2a22faf5c73607a84ec96297835e90330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T08:49:52.065 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:51 smithi179 podman[74027]: 2024-05-27 08:49:51.793552557 +0000 UTC m=+0.667766479 container died eb05b43e7c58e91fd2e6a0f7c3edd2a2a22faf5c73607a84ec96297835e90330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True) 2024-05-27T08:49:52.316 INFO:journalctl@ceph.osd.2.smithi179.stdout:May 27 08:49:52 smithi179 podman[74054]: 2024-05-27 08:49:52.29121425 +0000 UTC m=+0.486659406 container remove eb05b43e7c58e91fd2e6a0f7c3edd2a2a22faf5c73607a84ec96297835e90330 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:49:52.319 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.2.service' 2024-05-27T08:49:52.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:49:52.357 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-27T08:49:52.357 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-27T08:49:52.357 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.3 2024-05-27T08:49:52.711 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:52 smithi077 systemd[1]: Stopping Ceph osd.3 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:49:52.971 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3[53416]: 2024-05-27T08:49:52.709+0000 7fe5782f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:49:52.971 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3[53416]: 2024-05-27T08:49:52.709+0000 7fe5782f3700 -1 osd.3 99 *** Got signal Terminated *** 2024-05-27T08:49:52.971 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:52 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3[53416]: 2024-05-27T08:49:52.709+0000 7fe5782f3700 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:49:56.379 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:56 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3[53416]: 2024-05-27T08:49:56.019+0000 7fe5708df700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.886751+0000 front 2024-05-27T08:49:31.886727+0000 (oldest deadline 2024-05-27T08:49:55.386891+0000) 2024-05-27T08:49:56.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:49:56 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:49:56.196+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:49:56.879 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:49:56 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:49:56.547+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:49:57.378 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:49:57 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:49:57.215+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:49:57.379 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:57 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3[53416]: 2024-05-27T08:49:57.008+0000 7fe5708df700 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.886751+0000 front 2024-05-27T08:49:31.886727+0000 (oldest deadline 2024-05-27T08:49:55.386891+0000) 2024-05-27T08:49:57.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:49:57 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:49:57.133+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:49:57.878 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:49:57 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:49:57.593+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:49:58.478 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:58 smithi077 podman[103479]: 2024-05-27 08:49:58.18481001 +0000 UTC m=+5.622951178 container died 63e635a2bd110659e2d20c7d1e721af0fbbda19234121c6ee0c5639b3bd5e86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:49:58.479 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:49:58 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:49:58.259+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:49:58.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:49:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:49:58.131+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:49:58.803 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:49:58 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:49:58.586+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:49:58.803 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:58 smithi077 podman[103479]: 2024-05-27 08:49:58.58595818 +0000 UTC m=+6.024099348 container cleanup 63e635a2bd110659e2d20c7d1e721af0fbbda19234121c6ee0c5639b3bd5e86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:49:58.804 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:58 smithi077 bash[103479]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3 2024-05-27T08:49:59.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:49:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:49:58.822+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:49:59.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:49:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:49:58.643+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:49:59.128 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:58 smithi077 podman[103510]: 2024-05-27 08:49:58.803797843 +0000 UTC m=+0.608926145 container remove 63e635a2bd110659e2d20c7d1e721af0fbbda19234121c6ee0c5639b3bd5e86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-27T08:49:59.456 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:49:59 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:49:59.236+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:49:59.457 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:59 smithi077 podman[103598]: 2024-05-27 08:49:59.129725874 +0000 UTC m=+0.101899796 container create 74d6450db5276f93807c7d294cc3fa08b6ba494544d202f8b445578f6144910e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240521) 2024-05-27T08:49:59.457 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:59 smithi077 podman[103598]: 2024-05-27 08:49:59.044767543 +0000 UTC m=+0.016941485 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:49:59.457 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:59 smithi077 podman[103598]: 2024-05-27 08:49:59.374785403 +0000 UTC m=+0.346959350 container init 74d6450db5276f93807c7d294cc3fa08b6ba494544d202f8b445578f6144910e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2) 2024-05-27T08:49:59.457 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:59 smithi077 podman[103598]: 2024-05-27 08:49:59.378132087 +0000 UTC m=+0.350306016 container start 74d6450db5276f93807c7d294cc3fa08b6ba494544d202f8b445578f6144910e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T08:49:59.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:49:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:49:59.152+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:49:59.879 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:49:59 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:49:59.559+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:49:59.880 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:59 smithi077 podman[103598]: 2024-05-27 08:49:59.457035455 +0000 UTC m=+0.429209384 container attach 74d6450db5276f93807c7d294cc3fa08b6ba494544d202f8b445578f6144910e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-27T08:49:59.880 INFO:journalctl@ceph.osd.3.smithi077.stdout:May 27 08:49:59 smithi077 podman[103598]: 2024-05-27 08:49:59.674746976 +0000 UTC m=+0.646920914 container died 74d6450db5276f93807c7d294cc3fa08b6ba494544d202f8b445578f6144910e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:50:00.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:49:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:49:59.779+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:00.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:49:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:49:59.655+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:00.260 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.3.service' 2024-05-27T08:50:00.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:50:00.300 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-27T08:50:00.300 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-27T08:50:00.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.4 2024-05-27T08:50:00.579 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:00 smithi092 systemd[1]: Stopping Ceph osd.4 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:50:00.591 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:00 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:00.253+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:00.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:00 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:00.132+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:00.844 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:00 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4[47571]: 2024-05-27T08:50:00.578+0000 7faaeacd8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:50:00.844 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:00 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4[47571]: 2024-05-27T08:50:00.578+0000 7faaeacd8700 -1 osd.4 99 *** Got signal Terminated *** 2024-05-27T08:50:00.845 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:00 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4[47571]: 2024-05-27T08:50:00.578+0000 7faaeacd8700 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:50:00.878 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:00 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:00.591+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:01.098 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:00 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:00.779+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:01.099 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:00 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:00.644+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:01.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:00 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:00.843+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:01.355 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:01 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:01.098+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:01.586 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:01 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:01.245+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:01.878 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:01 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:01.586+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:02.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:01 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:01.733+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:02.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:01 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:01.641+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:02.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:01 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:01.818+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:02.570 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:02 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:02.294+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:02.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:02 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:02.128+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:02.878 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:02 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:02.570+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:03.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:02 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:02.772+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:03.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:02 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:02.674+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:03.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:02 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:02.796+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:03.596 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:03 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:03.283+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:03.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:03 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:03.147+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:03.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:03 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:03.461+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:03.878 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:03 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:03.596+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:03.879 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:03 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:03.596+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:04.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:03 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:03.795+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:04.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:03 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:03.641+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:04.235 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:03 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4[47571]: 2024-05-27T08:50:03.946+0000 7faae72cc700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:38.774115+0000 front 2024-05-27T08:49:38.774165+0000 (oldest deadline 2024-05-27T08:50:03.474291+0000) 2024-05-27T08:50:04.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:03 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:03.836+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:04.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:04 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:04.164+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:04.615 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:04 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:04.285+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:04.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:04 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:04.480+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:04.878 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:04 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:04.615+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:04.879 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:04 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:04.615+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:05.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:04 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:04.805+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:05.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:04 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:04.636+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:05.235 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:04 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4[47571]: 2024-05-27T08:50:04.953+0000 7faae72cc700 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:38.774115+0000 front 2024-05-27T08:49:38.774165+0000 (oldest deadline 2024-05-27T08:50:03.474291+0000) 2024-05-27T08:50:05.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:04 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:04.851+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:05.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:05 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:05.166+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:05.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:05 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:05.295+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:05.894 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:05 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:05.506+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:06.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:05 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:05.847+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:06.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:05 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:05.674+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:06.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:05 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:05.674+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:06.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:05 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:05.645+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:06.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:05 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:05.646+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:06.230 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:05 smithi092 podman[73638]: 2024-05-27 08:50:05.926716124 +0000 UTC m=+5.441993905 container died b243d4ef621d26baf0deeebec052d3140af1c59282cc812334b904334c891ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-27T08:50:06.230 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:05 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:05.892+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:06.485 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:06 smithi092 podman[73638]: 2024-05-27 08:50:06.312984926 +0000 UTC m=+5.828262703 container cleanup b243d4ef621d26baf0deeebec052d3140af1c59282cc812334b904334c891ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:50:06.485 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:06 smithi092 bash[73638]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4 2024-05-27T08:50:06.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:06 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:06.147+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:06.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:06 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:06.249+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:06.862 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:06 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:06.509+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:06.863 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:06 smithi092 podman[73654]: 2024-05-27 08:50:06.54803482 +0000 UTC m=+0.616994569 container remove b243d4ef621d26baf0deeebec052d3140af1c59282cc812334b904334c891ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-27T08:50:07.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:06 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:06.894+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:07.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:06 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:06.709+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:07.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:06 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:06.709+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:07.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:06 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:06.689+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:07.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:06 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:06.689+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:07.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:06 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:06.914+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:07.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:06 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:06.914+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:07.235 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:06 smithi092 podman[73748]: 2024-05-27 08:50:06.76593952 +0000 UTC m=+0.017766112 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:50:07.236 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:06 smithi092 podman[73748]: 2024-05-27 08:50:06.873863855 +0000 UTC m=+0.125690492 container create 1b115d93456ac039d951ae6d0d9d140bcccdec047ce04ed2b71203c0c7f98835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-05-27T08:50:07.236 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:07 smithi092 podman[73748]: 2024-05-27 08:50:07.076399532 +0000 UTC m=+0.328226123 container init 1b115d93456ac039d951ae6d0d9d140bcccdec047ce04ed2b71203c0c7f98835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS) 2024-05-27T08:50:07.236 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:07 smithi092 podman[73748]: 2024-05-27 08:50:07.079548148 +0000 UTC m=+0.331374739 container start 1b115d93456ac039d951ae6d0d9d140bcccdec047ce04ed2b71203c0c7f98835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-27T08:50:07.606 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:07 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:07.157+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:07.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:07 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:07.234+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:07.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:07 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:07.234+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:07.656 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:07 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:07.523+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:07.656 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:07 smithi092 podman[73748]: 2024-05-27 08:50:07.320240128 +0000 UTC m=+0.572066731 container attach 1b115d93456ac039d951ae6d0d9d140bcccdec047ce04ed2b71203c0c7f98835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:50:07.656 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:07 smithi092 podman[73748]: 2024-05-27 08:50:07.369115297 +0000 UTC m=+0.620941889 container died 1b115d93456ac039d951ae6d0d9d140bcccdec047ce04ed2b71203c0c7f98835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T08:50:07.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.4.service' 2024-05-27T08:50:07.928 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:07 smithi092 podman[73774]: 2024-05-27 08:50:07.8902505 +0000 UTC m=+0.512143302 container remove 1b115d93456ac039d951ae6d0d9d140bcccdec047ce04ed2b71203c0c7f98835 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-05-27T08:50:07.928 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:07 smithi092 systemd[1]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.4.service: Deactivated successfully. 2024-05-27T08:50:07.929 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:07 smithi092 systemd[1]: Stopped Ceph osd.4 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:50:07.929 INFO:journalctl@ceph.osd.4.smithi092.stdout:May 27 08:50:07 smithi092 systemd[1]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.4.service: Consumed 22.131s CPU time. 2024-05-27T08:50:07.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:50:07.943 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-27T08:50:07.944 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-27T08:50:07.944 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.5 2024-05-27T08:50:07.969 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:07 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:07.878+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:07.970 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:07 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:07.683+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:07.970 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:07 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:07.683+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:08.128 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:07 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:07.729+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:08.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:07 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:07.729+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:08.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:07 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:07.957+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:08.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:07 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:07.957+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:08.355 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:07 smithi179 systemd[1]: Stopping Ceph osd.5 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:50:08.356 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:08.206+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:08.356 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:08.206+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.965857+0000 front 2024-05-27T08:49:41.965887+0000 (oldest deadline 2024-05-27T08:50:07.866024+0000) 2024-05-27T08:50:08.356 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:08.217+0000 7f5830c2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:50:08.356 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:08.217+0000 7f5830c2b700 -1 osd.5 99 *** Got signal Terminated *** 2024-05-27T08:50:08.356 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:08.217+0000 7f5830c2b700 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:50:08.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:08.245+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:08.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:08.245+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:08.965 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:08 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:08.533+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:09.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:08.843+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:09.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:08.688+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:09.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:08 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:08.689+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:09.128 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:08.733+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:09.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:08.733+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:09.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:08 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:08.965+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:09.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:08 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:08.965+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:09.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:09.219+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:09.606 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:09.219+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.965857+0000 front 2024-05-27T08:49:41.965887+0000 (oldest deadline 2024-05-27T08:50:07.866024+0000) 2024-05-27T08:50:09.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:09.226+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:09.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:09.226+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:09.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:09 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:09.520+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:10.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:09.827+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:10.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:09.666+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:10.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:09 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:09.666+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:10.128 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:09.738+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:10.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:09 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:09.738+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:10.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:10.006+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:10.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:10.006+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:10.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:10.206+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:10.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:10.206+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.965857+0000 front 2024-05-27T08:49:41.965887+0000 (oldest deadline 2024-05-27T08:50:07.866024+0000) 2024-05-27T08:50:10.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:10 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:10.185+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:10.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:10 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:10.185+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:10.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:10.563+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:11.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:10.876+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:11.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:10.695+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:11.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:10 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:10.695+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:11.128 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:10 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:10.751+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:11.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:10 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:10.751+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:11.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:10.993+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:11.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:10 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:10.993+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:11.608 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:11.233+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:11.608 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:11.233+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.965857+0000 front 2024-05-27T08:49:41.965887+0000 (oldest deadline 2024-05-27T08:50:07.866024+0000) 2024-05-27T08:50:11.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:11.170+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:11.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:11.170+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:11.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:11 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:11.540+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:12.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:11.891+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:12.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:11.650+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:12.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:11 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:11.650+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:12.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:11.773+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:12.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:11.773+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:12.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:12.039+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:12.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:12.039+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:12.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:12.039+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:12.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:12.225+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:32.663656+0000 front 2024-05-27T08:49:32.663679+0000 (oldest deadline 2024-05-27T08:49:56.763860+0000) 2024-05-27T08:50:12.605 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5[48416]: 2024-05-27T08:50:12.225+0000 7f5829217700 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.965857+0000 front 2024-05-27T08:49:41.965887+0000 (oldest deadline 2024-05-27T08:50:07.866024+0000) 2024-05-27T08:50:12.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:12.159+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:12.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:12.159+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:12.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:12.159+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:12.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:12 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:12.514+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:13.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:12.867+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:13.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:12.662+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:13.106 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:12 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:12.662+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:13.117 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:12.771+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:13.117 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:12.771+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:13.117 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:12 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:12.771+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:13.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:13.117+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:13.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:13.117+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:13.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:13.117+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:13.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:13.065+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:13.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:13.065+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:13.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:13.065+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:13.901 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:13 smithi179 podman[74135]: 2024-05-27 08:50:13.621181881 +0000 UTC m=+5.480839217 container died 2ab3d9f9b8d7394efe3fb1e652bc6afb49c8d974771e75ddd47e0401d5eac593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=) 2024-05-27T08:50:13.901 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:13.854+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:13.902 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:13.616+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:13.902 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:13.616+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:13.902 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:13 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:13.616+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:13.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:13 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:13.505+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:14.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:13.775+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:14.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:13.775+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:14.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:13 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:13.775+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:14.358 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:13 smithi179 podman[74135]: 2024-05-27 08:50:13.901686282 +0000 UTC m=+5.761343609 container cleanup 2ab3d9f9b8d7394efe3fb1e652bc6afb49c8d974771e75ddd47e0401d5eac593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-27T08:50:14.358 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:13 smithi179 bash[74135]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5 2024-05-27T08:50:14.358 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:14 smithi179 podman[74150]: 2024-05-27 08:50:14.136011723 +0000 UTC m=+0.517085394 container remove 2ab3d9f9b8d7394efe3fb1e652bc6afb49c8d974771e75ddd47e0401d5eac593 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T08:50:14.460 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:14.016+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:14.460 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:14.016+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:14.461 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:14.016+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:14.616 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:14 smithi179 podman[74242]: 2024-05-27 08:50:14.382895192 +0000 UTC m=+0.017225587 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:50:14.617 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:14 smithi179 podman[74242]: 2024-05-27 08:50:14.487154784 +0000 UTC m=+0.121485206 container create b2f13374eb3199e33f32611979dc3c6bf72f9bf5f65483a89eab0d3e0d6d8416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-27T08:50:14.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:14.144+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:14.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:14.144+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:14.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:14.144+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:14.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:14.460+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:14.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:14 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:14.460+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:15.003 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:14.866+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:15.004 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:14.616+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:15.004 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:14.616+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:15.004 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:14 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:14.616+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:15.004 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:14 smithi179 podman[74242]: 2024-05-27 08:50:14.714457504 +0000 UTC m=+0.348787878 container init b2f13374eb3199e33f32611979dc3c6bf72f9bf5f65483a89eab0d3e0d6d8416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:50:15.004 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:14 smithi179 podman[74242]: 2024-05-27 08:50:14.717421183 +0000 UTC m=+0.351751564 container start b2f13374eb3199e33f32611979dc3c6bf72f9bf5f65483a89eab0d3e0d6d8416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=) 2024-05-27T08:50:15.005 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:14 smithi179 podman[74242]: 2024-05-27 08:50:14.789336263 +0000 UTC m=+0.423666678 container attach b2f13374eb3199e33f32611979dc3c6bf72f9bf5f65483a89eab0d3e0d6d8416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-27T08:50:15.107 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:14.810+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:15.107 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:14.810+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:15.107 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:14 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:14.810+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:15.267 INFO:journalctl@ceph.osd.5.smithi179.stdout:May 27 08:50:15 smithi179 podman[74242]: 2024-05-27 08:50:15.00425804 +0000 UTC m=+0.638588422 container died b2f13374eb3199e33f32611979dc3c6bf72f9bf5f65483a89eab0d3e0d6d8416 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:50:15.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:15.107+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:15.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:15.107+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:15.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:15.107+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:15.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:15.065+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:15.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:15.065+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:15.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:15.065+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:15.570 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.5.service' 2024-05-27T08:50:15.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:50:15.649 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-27T08:50:15.649 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-27T08:50:15.649 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.6 2024-05-27T08:50:15.822 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:15.569+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:15.823 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:15.569+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:15.823 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:15.569+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:15.954 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:15 smithi077 systemd[1]: Stopping Ceph osd.6 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:50:15.955 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:15.796+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:15.955 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:15.796+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:15.955 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:15.796+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:15.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:15.501+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:15.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:15 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:15.501+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:16.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:15.821+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:16.106 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:15 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:15.821+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:16.378 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:16.114+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:16.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:16.114+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:16.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:16.114+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:16.379 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:15.954+0000 7f2138fc9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:50:16.379 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:15.954+0000 7f2138fc9700 -1 osd.6 99 *** Got signal Terminated *** 2024-05-27T08:50:16.379 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:15 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:15.954+0000 7f2138fc9700 -1 osd.6 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:50:16.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:16 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:16.095+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:16.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:16 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:16.095+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:16.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:16 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:16.095+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:16.847 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:16 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:16.537+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:16.847 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:16 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:16.537+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:16.848 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:16 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:16.537+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:16.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:16 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:16.528+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:16.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:16 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:16.528+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:17.077 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:16.781+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:17.077 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:16.781+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:17.077 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:16.781+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:17.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:16 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:16.846+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:17.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:16 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:16.846+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:17.378 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:17 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:17.077+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:17.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:17 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:17.077+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:17.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:17 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:17.077+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:17.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:17 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:17.063+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:17.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:17 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:17.063+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:17.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:17 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:17.063+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:17.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:17 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:17.511+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:17.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:17 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:17.511+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:17.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:17 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:17.511+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:17.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:17 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:17.486+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:17.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:17 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:17.486+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:18.049 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:17 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:17.769+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:18.050 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:17 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:17.769+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:18.050 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:17 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:17.769+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:18.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:17 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:17.874+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:18.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:17 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:17.874+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:18.378 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:18 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:18.049+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:18.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:18 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:18.049+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:18.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:18 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:18.049+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:18.468 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:18 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:18.037+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:18.468 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:18 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:18.037+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:18.468 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:18 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:18.037+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:18.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:18 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:18.468+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:18.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:18 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:18.468+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:18.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:18 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:18.500+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:18.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:18 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:18.500+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:18.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:18 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:18.500+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:19.128 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:19.020+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:19.129 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:19.020+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:19.129 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:19.020+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:19.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:18 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:18.818+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:19.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:18 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:18.818+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:19.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:18 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:18.818+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:19.130 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:18 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:18.818+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:52.712813+0000 front 2024-05-27T08:49:52.712842+0000 (oldest deadline 2024-05-27T08:50:18.612900+0000) 2024-05-27T08:50:19.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:18 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:18.909+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:19.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:18 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:18.909+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:19.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:18 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:18.909+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:19.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:19 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:19.053+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:19.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:19 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:19.053+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:19.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:19 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:19.053+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:19.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:19 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:19.468+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:19.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:19 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:19.468+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:19.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:19 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:19.468+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:19.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:19 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:19.499+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:19.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:19 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:19.499+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:19.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:19 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:19.499+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:20.128 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:19.810+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:20.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:19.810+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:20.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:19.810+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:20.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:19.810+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:52.712813+0000 front 2024-05-27T08:49:52.712842+0000 (oldest deadline 2024-05-27T08:50:18.612900+0000) 2024-05-27T08:50:20.129 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:19.989+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:20.129 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:19.989+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:20.130 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:19 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:19.989+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:20.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:19 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:19.873+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:20.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:19 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:19.873+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:20.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:19 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:19.873+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:20.476 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:20 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:20.045+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:20.476 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:20 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:20.045+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:20.477 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:20 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:20.045+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:20.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:20 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:20.476+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:20.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:20 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:20.476+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:20.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:20 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:20.476+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:20.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:20 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:20.450+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:20.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:20 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:20.450+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:20.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:20 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:20.450+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:21.129 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:20 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:20.964+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:21.129 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:20 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:20.964+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:21.129 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:20 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:20.964+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:21.129 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:20 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:20.964+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:21.129 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:20 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:20.796+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.509498+0000 front 2024-05-27T08:49:30.509529+0000 (oldest deadline 2024-05-27T08:49:56.409604+0000) 2024-05-27T08:50:21.130 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:20 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:20.796+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:36.410305+0000 front 2024-05-27T08:49:36.410214+0000 (oldest deadline 2024-05-27T08:50:02.810955+0000) 2024-05-27T08:50:21.130 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:20 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:20.796+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.812692+0000 front 2024-05-27T08:49:49.812644+0000 (oldest deadline 2024-05-27T08:50:12.712421+0000) 2024-05-27T08:50:21.130 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:20 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6[60193]: 2024-05-27T08:50:20.796+0000 7f21315b5700 -1 osd.6 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:52.712813+0000 front 2024-05-27T08:49:52.712842+0000 (oldest deadline 2024-05-27T08:50:18.612900+0000) 2024-05-27T08:50:21.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:20 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:20.898+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:21.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:20 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:20.898+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:21.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:20 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:20.898+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:21.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:21.009+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:21.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:21.009+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:21.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:21.009+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:21.564 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:21 smithi077 podman[103704]: 2024-05-27 08:50:21.296382359 +0000 UTC m=+5.451776290 container died a1a78094f4073a369a7979d439543a889a75ed9c49c991192a2d82b7557e31ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-27T08:50:21.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:21 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:21.446+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:21.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:21 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:21.446+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:21.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:21 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:21.446+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:21.878 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:21 smithi077 podman[103704]: 2024-05-27 08:50:21.654398751 +0000 UTC m=+5.809792683 container cleanup a1a78094f4073a369a7979d439543a889a75ed9c49c991192a2d82b7557e31ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-27T08:50:21.879 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:21 smithi077 bash[103704]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6 2024-05-27T08:50:21.984 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:21.505+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:21.984 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:21.505+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:21.984 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:21.505+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:22.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:21.983+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:22.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:21.983+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:22.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:21 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:21.983+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:22.283 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:21 smithi077 podman[103720]: 2024-05-27 08:50:21.939316272 +0000 UTC m=+0.635231563 container remove a1a78094f4073a369a7979d439543a889a75ed9c49c991192a2d82b7557e31ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T08:50:22.283 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:22 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:22.013+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:22.284 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:22 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:22.013+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:22.284 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:22 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:22.013+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:22.284 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:22 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:22.013+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:22.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:21 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:21.896+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:22.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:21 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:21.896+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:22.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:21 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:21.896+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:22.583 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:22 smithi077 podman[103808]: 2024-05-27 08:50:22.186163723 +0000 UTC m=+0.017127894 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:50:22.584 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:22 smithi077 podman[103808]: 2024-05-27 08:50:22.290056879 +0000 UTC m=+0.121021046 container create d22b7294a38b9ea9bc49868da7433cc3bd6d6708c6ff0b22256c6ae4068bd76d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-05-27T08:50:22.584 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:22 smithi077 podman[103808]: 2024-05-27 08:50:22.501172548 +0000 UTC m=+0.332136714 container init d22b7294a38b9ea9bc49868da7433cc3bd6d6708c6ff0b22256c6ae4068bd76d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:50:22.584 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:22 smithi077 podman[103808]: 2024-05-27 08:50:22.504515536 +0000 UTC m=+0.335479694 container start d22b7294a38b9ea9bc49868da7433cc3bd6d6708c6ff0b22256c6ae4068bd76d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:50:22.584 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:22 smithi077 podman[103808]: 2024-05-27 08:50:22.584063133 +0000 UTC m=+0.415027301 container attach d22b7294a38b9ea9bc49868da7433cc3bd6d6708c6ff0b22256c6ae4068bd76d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-27T08:50:22.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:22 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:22.479+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:22.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:22 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:22.479+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:22.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:22 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:22.479+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:22.879 INFO:journalctl@ceph.osd.6.smithi077.stdout:May 27 08:50:22 smithi077 podman[103808]: 2024-05-27 08:50:22.806606705 +0000 UTC m=+0.637570863 container died d22b7294a38b9ea9bc49868da7433cc3bd6d6708c6ff0b22256c6ae4068bd76d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-05-27T08:50:22.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:22 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:22.555+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:22.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:22 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:22.555+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:22.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:22 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:22.555+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:23.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:22 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:22.938+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:23.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:22 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:22.938+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:23.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:22 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:22.938+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:23.362 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:22 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:22.997+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:23.362 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:22 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:22.998+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:23.363 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:22 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:22.998+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:23.363 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:22 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:22.998+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:23.383 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.6.service' 2024-05-27T08:50:23.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:50:23.422 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-27T08:50:23.422 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-27T08:50:23.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.7 2024-05-27T08:50:23.447 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:23.023+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:23.447 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:23.023+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:23.447 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:23.023+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:23.447 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:23.023+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:23.716 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:23 smithi092 systemd[1]: Stopping Ceph osd.7 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:50:23.716 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:23.586+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:23.716 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:23.586+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:23.716 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:23.586+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:23.717 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:23.715+0000 7fa38d229700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:50:23.717 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:23.715+0000 7fa38d229700 -1 osd.7 99 *** Got signal Terminated *** 2024-05-27T08:50:23.717 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:23 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:23.715+0000 7fa38d229700 -1 osd.7 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:50:23.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:23 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:23.439+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:23.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:23 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:23.439+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:23.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:23 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:23.439+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:24.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:23 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:23.954+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:24.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:23 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:23.954+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:24.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:23 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:23.954+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:24.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:24 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:24.021+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:24.382 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:24 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:24.021+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:24.383 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:24 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:24.021+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:24.383 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:24 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:24.021+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:24.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:24.026+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:24.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:24.026+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:24.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:24.026+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:24.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:24.026+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:24.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:24 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:24.429+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:24.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:24 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:24.429+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:24.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:24 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:24.429+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:24.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:24.631+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:24.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:24.631+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:24.986 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:24.631+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:25.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:24 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:24.982+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:25.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:24 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:24.982+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:25.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:24 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:24.982+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:25.378 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:25.069+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:25.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:25.069+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:25.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:25.069+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:25.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:25 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:25.069+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:25.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:25.075+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:25.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:25.075+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:25.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:25.075+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:25.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:25.075+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:25.713 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:25 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:25.383+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:25.714 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:25 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:25.383+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:25.714 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:25 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:25.383+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:25.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:25.623+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:25.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:25.623+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:25.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:25 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:25.623+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:26.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:25 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:25.947+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:26.105 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:25 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:25.947+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:26.106 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:25 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:25.947+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:26.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:26 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:26.071+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:26.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:26 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:26.071+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:26.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:26 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:26.071+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:26.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:26 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:26.071+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:26.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:26 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:26.046+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:26.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:26 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:26.046+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:26.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:26 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:26.046+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:26.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:26 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:26.046+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:26.605 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:26 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:26.359+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:26.605 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:26 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:26.359+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:26.606 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:26 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:26.359+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:26.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:26 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:26.590+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:27.014 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:26 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:26.590+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:27.014 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:26 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:26.590+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:27.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:26 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:26.975+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:27.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:26 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:26.975+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:27.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:26 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:26.975+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:27.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:27 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:27.118+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:27.380 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:27 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:27.118+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:27.380 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:27 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:27.118+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:27.380 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:27 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:27.118+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:27.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:27 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:27.087+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:27.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:27 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:27.087+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:27.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:27 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:27.087+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:27.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:27 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:27.087+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:27.859 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:27 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:27.407+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:27.859 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:27 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:27.407+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:27.859 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:27 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:27.407+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:27.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:27 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:27.574+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:27.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:27 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:27.574+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:27.986 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:27 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:27.574+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:27.986 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:27 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:27.574+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:01.153826+0000 front 2024-05-27T08:50:01.153866+0000 (oldest deadline 2024-05-27T08:50:27.053991+0000) 2024-05-27T08:50:28.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:27 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:27.938+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:28.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:27 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:27.938+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:28.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:27 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:27.938+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:28.378 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:28.112+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:28.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:28.112+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:28.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:28.112+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:28.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:28 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:28.112+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:28.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:28.131+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:28.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:28.131+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:28.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:28.131+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:28.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:28.131+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:28.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:28.131+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:28.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:28 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:28.394+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:28.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:28 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:28.394+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:28.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:28 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:28.394+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:28.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:28 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:28.394+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:28.985 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:28.554+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:39.550505+0000 front 2024-05-27T08:49:39.550480+0000 (oldest deadline 2024-05-27T08:50:03.050709+0000) 2024-05-27T08:50:28.986 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:28.554+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.352143+0000 front 2024-05-27T08:49:48.351976+0000 (oldest deadline 2024-05-27T08:50:13.651869+0000) 2024-05-27T08:50:28.986 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:28.554+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:53.652370+0000 front 2024-05-27T08:49:53.652411+0000 (oldest deadline 2024-05-27T08:50:18.952480+0000) 2024-05-27T08:50:28.986 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7[54423]: 2024-05-27T08:50:28.554+0000 7fa38981d700 -1 osd.7 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:01.153826+0000 front 2024-05-27T08:50:01.153866+0000 (oldest deadline 2024-05-27T08:50:27.053991+0000) 2024-05-27T08:50:29.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:28 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:28.931+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:29.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:28 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:28.931+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:29.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:28 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:28.931+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:29.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:28 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:28.931+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.881474+0000 front 2024-05-27T08:50:02.881403+0000 (oldest deadline 2024-05-27T08:50:28.781252+0000) 2024-05-27T08:50:29.378 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:29.113+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:29.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:29.113+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:29.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:29.113+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:29.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:29 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:29.113+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:29.425 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:29 smithi092 podman[73865]: 2024-05-27 08:50:29.055975786 +0000 UTC m=+5.437940920 container died 9614a9e301a8438724d5a75afd9f1ed2c1edd30f360f3c6198c7e440507391a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-27T08:50:29.425 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:29 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:29.173+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:29.425 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:29 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:29.173+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:29.425 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:29 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:29.173+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:29.425 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:29 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:29.173+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:29.426 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:29 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:29.173+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:29.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:29 smithi092 podman[73865]: 2024-05-27 08:50:29.540592145 +0000 UTC m=+5.922557270 container cleanup 9614a9e301a8438724d5a75afd9f1ed2c1edd30f360f3c6198c7e440507391a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5) 2024-05-27T08:50:29.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:29 smithi092 bash[73865]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7 2024-05-27T08:50:29.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:29 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:29.394+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:29.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:29 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:29.394+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:29.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:29 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:29.394+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:29.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:29 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:29.394+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:30.111 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:29 smithi092 podman[73882]: 2024-05-27 08:50:29.792539914 +0000 UTC m=+0.735117221 container remove 9614a9e301a8438724d5a75afd9f1ed2c1edd30f360f3c6198c7e440507391a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-27T08:50:30.346 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:29 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:29.927+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:30.346 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:29 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:29.927+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:30.346 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:29 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:29.927+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:30.346 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:29 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:29.927+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.881474+0000 front 2024-05-27T08:50:02.881403+0000 (oldest deadline 2024-05-27T08:50:28.781252+0000) 2024-05-27T08:50:30.403 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:30 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:30.198+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:30.403 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:30 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:30.198+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:30.404 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:30 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:30.198+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:30.404 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:30 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:30.198+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:30.404 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:30 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:30.198+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:30.404 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:30 smithi092 podman[73970]: 2024-05-27 08:50:30.014533626 +0000 UTC m=+0.016750166 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:50:30.404 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:30 smithi092 podman[73970]: 2024-05-27 08:50:30.118303951 +0000 UTC m=+0.120520500 container create b053c759f167cc6598b5e9b6a9d96d612173eea547d2176d195a0a07b759c4f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5) 2024-05-27T08:50:30.404 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:30 smithi092 podman[73970]: 2024-05-27 08:50:30.328887556 +0000 UTC m=+0.331104082 container init b053c759f167cc6598b5e9b6a9d96d612173eea547d2176d195a0a07b759c4f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T08:50:30.405 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:30 smithi092 podman[73970]: 2024-05-27 08:50:30.332446831 +0000 UTC m=+0.334663357 container start b053c759f167cc6598b5e9b6a9d96d612173eea547d2176d195a0a07b759c4f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:50:30.405 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:30 smithi092 podman[73970]: 2024-05-27 08:50:30.403971041 +0000 UTC m=+0.406187576 container attach b053c759f167cc6598b5e9b6a9d96d612173eea547d2176d195a0a07b759c4f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-27T08:50:30.605 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:30 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:30.345+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:30.605 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:30 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:30.345+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:30.606 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:30 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:30.345+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:30.606 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:30 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:30.345+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:30.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:30 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:30.140+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:30.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:30 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:30.140+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:30.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:30 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:30.140+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:30.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:30 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:30.140+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:30.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:30 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:30.140+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:30.735 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:30 smithi092 podman[73970]: 2024-05-27 08:50:30.623085777 +0000 UTC m=+0.625302304 container died b053c759f167cc6598b5e9b6a9d96d612173eea547d2176d195a0a07b759c4f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-27T08:50:31.158 INFO:journalctl@ceph.osd.7.smithi092.stdout:May 27 08:50:31 smithi092 podman[73995]: 2024-05-27 08:50:31.142272008 +0000 UTC m=+0.511626630 container remove b053c759f167cc6598b5e9b6a9d96d612173eea547d2176d195a0a07b759c4f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T08:50:31.162 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.7.service' 2024-05-27T08:50:31.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:50:31.240 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-27T08:50:31.241 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-27T08:50:31.241 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.8 2024-05-27T08:50:31.267 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:30 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:30.900+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:31.267 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:30 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:30.900+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:31.267 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:30 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:30.900+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:31.267 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:30 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:30.900+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.881474+0000 front 2024-05-27T08:50:02.881403+0000 (oldest deadline 2024-05-27T08:50:28.781252+0000) 2024-05-27T08:50:31.416 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:31 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:31.195+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:31.416 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:31 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:31.195+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:31.416 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:31 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:31.195+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:31.417 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:31 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:31.195+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:31.417 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:31 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:31.195+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:31.550 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:31 smithi179 systemd[1]: Stopping Ceph osd.8 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:50:31.550 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:31.385+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:31.551 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:31.385+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:31.551 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:31.385+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:31.551 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:31.385+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:31.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:31 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:31.180+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:31.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:31 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:31.180+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:31.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:31 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:31.180+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:31.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:31 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:31.180+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:31.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:31 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:31.180+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:31.855 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:31.549+0000 7f45c7319700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:50:31.855 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:31.549+0000 7f45c7319700 -1 osd.8 99 *** Got signal Terminated *** 2024-05-27T08:50:31.855 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:31.549+0000 7f45c7319700 -1 osd.8 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:50:32.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:31.924+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:32.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:31.924+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:32.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:31.924+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:32.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:31 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:31.924+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.881474+0000 front 2024-05-27T08:50:02.881403+0000 (oldest deadline 2024-05-27T08:50:28.781252+0000) 2024-05-27T08:50:32.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:32 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:32.148+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:32.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:32 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:32.148+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:32.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:32 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:32.148+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:32.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:32 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:32.148+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:32.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:32 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:32.148+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:32.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:32 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:32.167+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:32.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:32 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:32.167+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:32.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:32 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:32.167+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:32.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:32 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:32.167+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:32.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:32 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:32.167+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:32.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:32 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:32.364+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:32.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:32 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:32.364+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:32.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:32 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:32.364+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:32.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:32 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:32.364+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:33.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:32 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:32.960+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:33.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:32 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:32.960+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:33.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:32 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:32.960+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:33.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:32 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:32.960+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.881474+0000 front 2024-05-27T08:50:02.881403+0000 (oldest deadline 2024-05-27T08:50:28.781252+0000) 2024-05-27T08:50:33.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:33 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:33.159+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:33.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:33 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:33.159+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:33.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:33 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:33.159+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:33.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:33 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:33.159+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:33.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:33 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:33.159+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:33.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:33 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:33.154+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:33.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:33 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:33.154+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:33.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:33 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:33.154+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:33.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:33 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:33.154+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:33.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:33 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:33.154+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:33.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:33 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:33.405+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:33.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:33 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:33.405+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:33.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:33 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:33.405+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:33.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:33 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:33.405+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:34.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:33 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:33.966+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:34.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:33 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:33.966+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:34.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:33 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:33.966+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:34.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:33 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:33.966+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.881474+0000 front 2024-05-27T08:50:02.881403+0000 (oldest deadline 2024-05-27T08:50:28.781252+0000) 2024-05-27T08:50:34.378 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:34.196+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:34.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:34.196+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:34.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:34.196+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:34.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:34.196+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:34.379 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:34 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:34.196+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:34.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:34 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:34.165+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:34.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:34 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:34.165+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:34.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:34 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:34.165+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:34.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:34 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:34.165+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:34.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:34 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:34.165+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:34.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:34 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:34.453+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:34.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:34 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:34.453+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:34.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:34 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:34.454+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:34.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:34 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:34.454+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:35.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:35.005+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:35.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:35.005+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:35.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:35.005+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:35.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:35.005+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.881474+0000 front 2024-05-27T08:50:02.881403+0000 (oldest deadline 2024-05-27T08:50:28.781252+0000) 2024-05-27T08:50:35.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:35.005+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:08.781574+0000 front 2024-05-27T08:50:08.781606+0000 (oldest deadline 2024-05-27T08:50:34.081638+0000) 2024-05-27T08:50:35.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:35 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:35.152+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:35.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:35 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:35.152+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:35.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:35 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:35.152+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:35.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:35 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:35.152+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:35.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:35 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:35.152+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:35.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:35 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:35.152+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:35.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:35 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:35.241+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:35.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:35 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:35.241+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:35.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:35 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:35.241+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:35.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:35 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:35.241+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:35.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:35 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:35.241+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:35.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:35 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:35.241+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:35.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:35.456+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:35.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:35.456+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:35.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:35.456+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:35.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:35 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:35.456+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:36.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:36.053+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.976062+0000 front 2024-05-27T08:49:31.976093+0000 (oldest deadline 2024-05-27T08:49:57.876340+0000) 2024-05-27T08:50:36.355 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:36.053+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:44.277809+0000 front 2024-05-27T08:49:44.277997+0000 (oldest deadline 2024-05-27T08:50:14.878639+0000) 2024-05-27T08:50:36.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:36.053+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.379668+0000 front 2024-05-27T08:49:55.379630+0000 (oldest deadline 2024-05-27T08:50:18.279685+0000) 2024-05-27T08:50:36.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:36.053+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.881474+0000 front 2024-05-27T08:50:02.881403+0000 (oldest deadline 2024-05-27T08:50:28.781252+0000) 2024-05-27T08:50:36.356 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8[55380]: 2024-05-27T08:50:36.053+0000 7f45c390d700 -1 osd.8 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:08.781574+0000 front 2024-05-27T08:50:08.781606+0000 (oldest deadline 2024-05-27T08:50:34.081638+0000) 2024-05-27T08:50:36.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:36 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:36.145+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:36.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:36 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:36.145+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:36.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:36 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:36.145+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:36.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:36 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:36.145+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:36.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:36 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:36.145+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:36.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:36 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:36.145+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:36.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:36 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:36.263+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:36.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:36 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:36.263+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:36.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:36 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:36.263+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:36.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:36 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:36.263+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:36.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:36 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:36.263+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:36.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:36 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:36.263+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:36.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:36.447+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:36.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:36.447+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:36.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:36.447+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:36.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:36.447+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:37.357 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:37 smithi179 podman[74363]: 2024-05-27 08:50:37.076250981 +0000 UTC m=+5.635630866 container died 5e04a1d44554bde73f1da10ad8a0b336db41df735e2595372695727d7d222b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521) 2024-05-27T08:50:37.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:37.134+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:37.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:37.134+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:37.487 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:37.134+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:37.487 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:37.134+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:37.487 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:37.134+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:37.487 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:37 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:37.134+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:37.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:37 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:37.256+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:37.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:37 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:37.256+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:37.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:37 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:37.256+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:37.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:37 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:37.256+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:37.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:37 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:37.256+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:37.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:37 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:37.256+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:37.858 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:37 smithi179 podman[74363]: 2024-05-27 08:50:37.386844818 +0000 UTC m=+5.946224700 container cleanup 5e04a1d44554bde73f1da10ad8a0b336db41df735e2595372695727d7d222b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:50:37.858 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:37 smithi179 bash[74363]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8 2024-05-27T08:50:37.859 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:37 smithi179 podman[74378]: 2024-05-27 08:50:37.571355577 +0000 UTC m=+0.493228170 container remove 5e04a1d44554bde73f1da10ad8a0b336db41df735e2595372695727d7d222b7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-27T08:50:37.859 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:37 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:37.442+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:37.859 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:37 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:37.442+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:37.859 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:37 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:37.442+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:37.860 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:37 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:37.442+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:37.860 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:37 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:37.442+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:38.274 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:38 smithi179 podman[74470]: 2024-05-27 08:50:37.91359901 +0000 UTC m=+0.017373730 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:50:38.275 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:38 smithi179 podman[74470]: 2024-05-27 08:50:38.03058113 +0000 UTC m=+0.134355839 container create b0e895152c6c3a6029301ab1dbbdd775deac1d75a43a316390d6e7b2146b0c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:50:38.275 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:38 smithi179 podman[74470]: 2024-05-27 08:50:38.275027217 +0000 UTC m=+0.378801933 container init b0e895152c6c3a6029301ab1dbbdd775deac1d75a43a316390d6e7b2146b0c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-27T08:50:38.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:38 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:38.084+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:38.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:38 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:38.084+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:38.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:38 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:38.084+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:38.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:38 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:38.084+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:38.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:38 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:38.084+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:38.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:38 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:38.084+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:38.571 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:38 smithi179 podman[74470]: 2024-05-27 08:50:38.278701826 +0000 UTC m=+0.382476542 container start b0e895152c6c3a6029301ab1dbbdd775deac1d75a43a316390d6e7b2146b0c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-27T08:50:38.571 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:38 smithi179 podman[74470]: 2024-05-27 08:50:38.357814796 +0000 UTC m=+0.461589503 container attach b0e895152c6c3a6029301ab1dbbdd775deac1d75a43a316390d6e7b2146b0c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:50:38.571 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:38 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:38.433+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:38.572 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:38 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:38.433+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:38.572 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:38 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:38.433+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:38.572 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:38 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:38.433+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:38.572 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:38 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:38.433+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:38.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:38 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:38.295+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:38.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:38 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:38.295+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:38.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:38 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:38.295+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:38.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:38 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:38.295+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:38.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:38 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:38.295+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:38.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:38 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:38.295+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:38.855 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:38 smithi179 podman[74470]: 2024-05-27 08:50:38.571598891 +0000 UTC m=+0.675373615 container died b0e895152c6c3a6029301ab1dbbdd775deac1d75a43a316390d6e7b2146b0c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521) 2024-05-27T08:50:39.121 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.8.service' 2024-05-27T08:50:39.184 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:39 smithi179 podman[74496]: 2024-05-27 08:50:39.103165727 +0000 UTC m=+0.525679697 container remove b0e895152c6c3a6029301ab1dbbdd775deac1d75a43a316390d6e7b2146b0c1e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, io.buildah.version=1.33.5) 2024-05-27T08:50:39.184 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:39 smithi179 systemd[1]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.8.service: Deactivated successfully. 2024-05-27T08:50:39.184 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:39 smithi179 systemd[1]: Stopped Ceph osd.8 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:50:39.184 INFO:journalctl@ceph.osd.8.smithi179.stdout:May 27 08:50:39 smithi179 systemd[1]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.8.service: Consumed 35.299s CPU time. 2024-05-27T08:50:39.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:50:39.200 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-27T08:50:39.201 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-27T08:50:39.201 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.9 2024-05-27T08:50:39.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.041+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:39.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.041+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:39.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.041+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:39.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.041+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:39.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.041+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:39.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.041+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:39.507 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 systemd[1]: Stopping Ceph osd.9 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:50:39.507 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:39.507 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:39.507 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:39.507 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:39.507 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:39.508 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:39.508 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.506+0000 7f762ac3d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:50:39.508 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.506+0000 7f762ac3d700 -1 osd.9 99 *** Got signal Terminated *** 2024-05-27T08:50:39.508 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:39 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:39.506+0000 7f762ac3d700 -1 osd.9 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:50:39.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:39 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:39.436+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:39.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:39 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:39.436+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:39.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:39 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:39.436+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:39.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:39 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:39.436+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:39.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:39 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:39.436+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:40.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.994+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:40.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.994+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:40.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.994+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:40.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.994+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:40.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.994+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:40.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:39 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:39.994+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:40.605 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:40 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:40.485+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:40.606 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:40 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:40.485+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:40.606 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:40 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:40.485+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:40.606 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:40 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:40.485+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:40.606 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:40 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:40.485+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:40.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:40 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:40.321+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:40.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:40 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:40.321+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:40.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:40 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:40.321+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:40.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:40 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:40.321+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:40.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:40 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:40.321+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:40.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:40 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:40.321+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:41.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:40 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:40.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:41.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:40 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:40.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:41.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:40 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:40.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:41.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:40 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:40.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:41.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:40 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:40.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:41.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:40 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:40.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:41.628 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:41 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:41.327+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:41.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:41 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:41.327+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:41.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:41 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:41.327+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:41.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:41 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:41.327+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:41.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:41 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:41.327+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:41.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:41 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:41.327+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:41.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:41 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:41.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:41.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:41 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:41.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:41.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:41 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:41.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:41.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:41 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:41.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:41.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:41 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:41.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:42.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:41 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:41.960+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:42.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:41 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:41.960+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:42.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:41 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:41.960+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:42.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:41 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:41.960+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:42.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:41 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:41.960+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:42.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:41 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:41.960+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:42.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:42 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:42.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:42.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:42 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:42.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:42.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:42 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:42.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:42.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:42 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:42.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:42.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:42 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:42.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:42.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:42 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:42.342+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:42.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:42 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:42.523+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:42.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:42 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:42.523+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:42.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:42 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:42.523+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:42.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:42 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:42.523+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:42.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:42 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:42.523+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:42.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:42 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:42.523+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:43.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:42 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:42.991+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:43.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:42 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:42.991+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:43.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:42 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:42.991+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:43.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:42 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:42.991+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:43.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:42 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:42.991+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:43.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:42 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:42.991+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:43.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:43 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:43.546+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:43.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:43 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:43.546+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:43.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:43 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:43.546+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:43.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:43 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:43.546+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:43.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:43 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:43.546+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:43.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:43 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:43.546+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:43.878 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:43 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:43.386+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:43.879 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:43 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:43.386+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:43.879 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:43 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:43.386+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:43.879 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:43 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:43.386+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:43.879 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:43 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:43.386+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:43.879 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:43 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:43.386+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:44.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:43 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:43.973+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:44.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:43 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:43.973+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:44.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:43 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:43.973+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:44.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:43 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:43.973+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:44.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:43 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:43.973+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:44.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:43 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:43.973+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:44.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:43 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:43.973+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:44.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:44 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:44.367+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:31.989659+0000 front 2024-05-27T08:49:31.989642+0000 (oldest deadline 2024-05-27T08:49:56.089767+0000) 2024-05-27T08:50:44.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:44 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:44.367+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:41.891488+0000 front 2024-05-27T08:49:41.891457+0000 (oldest deadline 2024-05-27T08:50:07.191383+0000) 2024-05-27T08:50:44.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:44 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:44.367+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:47.191925+0000 front 2024-05-27T08:49:47.192149+0000 (oldest deadline 2024-05-27T08:50:11.291935+0000) 2024-05-27T08:50:44.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:44 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:44.367+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:55.992844+0000 front 2024-05-27T08:49:55.992813+0000 (oldest deadline 2024-05-27T08:50:20.692902+0000) 2024-05-27T08:50:44.630 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:44 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:44.367+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:04.794082+0000 front 2024-05-27T08:50:04.794191+0000 (oldest deadline 2024-05-27T08:50:29.493896+0000) 2024-05-27T08:50:44.631 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:44 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9[67033]: 2024-05-27T08:50:44.367+0000 7f7623229700 -1 osd.9 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.595000+0000 front 2024-05-27T08:50:10.595031+0000 (oldest deadline 2024-05-27T08:50:35.195335+0000) 2024-05-27T08:50:44.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:44.533+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:44.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:44.533+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:44.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:44.533+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:44.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:44.533+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:44.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:44.533+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:44.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:44.533+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:45.128 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:44 smithi077 podman[103929]: 2024-05-27 08:50:44.852633308 +0000 UTC m=+5.450317177 container died 3aadfe1dd7449dae33cd69a0faf2026911e092b1f90e2a8ee8793aaba9f73b34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-27T08:50:45.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:44 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:44.937+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:45.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:44 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:44.937+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:45.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:44 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:44.937+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:45.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:44 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:44.937+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:45.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:44 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:44.937+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:45.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:44 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:44.937+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:45.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:44 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:44.937+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:45.624 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:45 smithi077 podman[103929]: 2024-05-27 08:50:45.323069123 +0000 UTC m=+5.920752979 container cleanup 3aadfe1dd7449dae33cd69a0faf2026911e092b1f90e2a8ee8793aaba9f73b34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=) 2024-05-27T08:50:45.625 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:45 smithi077 bash[103929]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9 2024-05-27T08:50:45.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:45 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:45.558+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:45.858 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:45 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:45.558+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:45.858 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:45 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:45.558+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:45.858 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:45 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:45.558+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:45.858 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:45 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:45.558+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:45.859 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:45 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:45.558+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:45.878 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:45 smithi077 podman[103946]: 2024-05-27 08:50:45.624466337 +0000 UTC m=+0.771975686 container remove 3aadfe1dd7449dae33cd69a0faf2026911e092b1f90e2a8ee8793aaba9f73b34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-27T08:50:46.236 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:45 smithi077 podman[104036]: 2024-05-27 08:50:45.842156127 +0000 UTC m=+0.017047906 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:50:46.236 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:45 smithi077 podman[104036]: 2024-05-27 08:50:45.950355943 +0000 UTC m=+0.125247704 container create abfa02114d6f44f8353945eb062f48afd0c7a3b87aae9f3dfbfc1cd77084a28a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:50:46.236 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:46 smithi077 podman[104036]: 2024-05-27 08:50:46.161163074 +0000 UTC m=+0.336054844 container init abfa02114d6f44f8353945eb062f48afd0c7a3b87aae9f3dfbfc1cd77084a28a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux ) 2024-05-27T08:50:46.236 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:46 smithi077 podman[104036]: 2024-05-27 08:50:46.164449408 +0000 UTC m=+0.339341176 container start abfa02114d6f44f8353945eb062f48afd0c7a3b87aae9f3dfbfc1cd77084a28a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-27T08:50:46.236 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:46 smithi077 podman[104036]: 2024-05-27 08:50:46.236218729 +0000 UTC m=+0.411110488 container attach abfa02114d6f44f8353945eb062f48afd0c7a3b87aae9f3dfbfc1cd77084a28a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, ceph=True) 2024-05-27T08:50:46.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:45 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:45.986+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:46.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:45 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:45.986+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:46.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:45 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:45.986+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:46.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:45 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:45.986+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:46.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:45 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:45.986+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:46.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:45 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:45.986+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:46.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:45 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:45.986+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:46.629 INFO:journalctl@ceph.osd.9.smithi077.stdout:May 27 08:50:46 smithi077 podman[104036]: 2024-05-27 08:50:46.455555928 +0000 UTC m=+0.630447685 container died abfa02114d6f44f8353945eb062f48afd0c7a3b87aae9f3dfbfc1cd77084a28a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-05-27T08:50:46.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:46 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:46.554+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:46.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:46 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:46.554+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:46.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:46 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:46.554+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:46.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:46 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:46.554+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:46.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:46 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:46.554+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:46.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:46 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:46.554+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:47.016 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.9.service' 2024-05-27T08:50:47.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:50:47.056 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-27T08:50:47.056 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-27T08:50:47.056 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.10 2024-05-27T08:50:47.369 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.013+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:47.370 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.013+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:47.370 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.013+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:47.370 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.013+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:47.370 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.013+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:47.370 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.013+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:47.370 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.013+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:47.371 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 systemd[1]: Stopping Ceph osd.10 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:50:47.371 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.368+0000 7f84a2525700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:50:47.371 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.368+0000 7f84a2525700 -1 osd.10 99 *** Got signal Terminated *** 2024-05-27T08:50:47.371 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.368+0000 7f84a2525700 -1 osd.10 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:50:47.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:47 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:47.516+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:47.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:47 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:47.516+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:47.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:47 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:47.516+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:47.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:47 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:47.516+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:47.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:47 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:47.516+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:47.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:47 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:47.516+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:48.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.997+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:48.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.997+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:48.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.997+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:48.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.997+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:48.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.997+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:48.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.997+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:48.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:47 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:47.997+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:48.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:48 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:48.477+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:48.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:48 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:48.477+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:48.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:48 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:48.477+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:48.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:48 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:48.477+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:48.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:48 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:48.477+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:48.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:48 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:48.477+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:49.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:48 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:48.995+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:49.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:48 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:48.995+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:49.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:48 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:48.995+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:49.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:48 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:48.995+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:49.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:48 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:48.995+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:49.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:48 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:48.995+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:49.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:48 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:48.995+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:49.807 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:49 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:49.515+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:49.808 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:49 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:49.515+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:49.808 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:49 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:49.515+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:49.808 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:49 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:49.515+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:49.808 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:49 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:49.515+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:49.808 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:49 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:49.515+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:50.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.042+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:50.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.042+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:50.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.042+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:50.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.042+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:50.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.042+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:50.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.042+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:50.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.042+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:50.858 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:50 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:50.525+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:50.954 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:50 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:50.525+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:50.954 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:50 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:50.525+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:50.955 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:50 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:50.525+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:50.955 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:50 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:50.525+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:50.955 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:50 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:50.525+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:51.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:51.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:51.485 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:51.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:51.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:51.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:51.486 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:50 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:50.992+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:51.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:51 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:51.527+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:51.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:51 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:51.527+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:51.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:51 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:51.527+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:51.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:51 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:51.527+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:51.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:51 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:51.527+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:51.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:51 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:51.527+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:52.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:51 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:51.984+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:30.304648+0000 front 2024-05-27T08:49:30.304657+0000 (oldest deadline 2024-05-27T08:50:00.305093+0000) 2024-05-27T08:50:52.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:51 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:51.984+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:42.006246+0000 front 2024-05-27T08:49:42.006269+0000 (oldest deadline 2024-05-27T08:50:06.706456+0000) 2024-05-27T08:50:52.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:51 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:51.984+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:49.607479+0000 front 2024-05-27T08:49:49.607498+0000 (oldest deadline 2024-05-27T08:50:11.808017+0000) 2024-05-27T08:50:52.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:51 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:51.984+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6814 osd.3 since back 2024-05-27T08:49:56.810808+0000 front 2024-05-27T08:49:56.810892+0000 (oldest deadline 2024-05-27T08:50:22.710989+0000) 2024-05-27T08:50:52.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:51 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:51.984+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:02.711548+0000 front 2024-05-27T08:50:02.711524+0000 (oldest deadline 2024-05-27T08:50:27.411628+0000) 2024-05-27T08:50:52.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:51 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:51.984+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.213012+0000 front 2024-05-27T08:50:10.212913+0000 (oldest deadline 2024-05-27T08:50:34.912684+0000) 2024-05-27T08:50:52.236 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:51 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10[61447]: 2024-05-27T08:50:51.984+0000 7f849ab11700 -1 osd.10 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:17.114246+0000 front 2024-05-27T08:50:17.114262+0000 (oldest deadline 2024-05-27T08:50:43.514489+0000) 2024-05-27T08:50:52.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:52 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:52.553+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:52.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:52 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:52.553+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:52.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:52 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:52.553+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:52.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:52 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:52.553+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:52.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:52 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:52.553+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:52.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:52 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:52.553+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:52.977 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:52 smithi092 podman[74087]: 2024-05-27 08:50:52.704005174 +0000 UTC m=+5.457059327 container stop 8bc1eb7f780e603533786bf27700cfa34b5af4354c97ceca7d9167b3058410d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:50:52.978 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:52 smithi092 podman[74087]: 2024-05-27 08:50:52.729688319 +0000 UTC m=+5.482742470 container died 8bc1eb7f780e603533786bf27700cfa34b5af4354c97ceca7d9167b3058410d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-27T08:50:53.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:53 smithi092 podman[74087]: 2024-05-27 08:50:53.07611839 +0000 UTC m=+5.829172531 container cleanup 8bc1eb7f780e603533786bf27700cfa34b5af4354c97ceca7d9167b3058410d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:50:53.235 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:53 smithi092 bash[74087]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10 2024-05-27T08:50:53.695 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:53 smithi092 podman[74102]: 2024-05-27 08:50:53.25309137 +0000 UTC m=+0.542412824 container remove 8bc1eb7f780e603533786bf27700cfa34b5af4354c97ceca7d9167b3058410d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240521) 2024-05-27T08:50:53.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:53 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:53.585+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:53.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:53 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:53.585+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:53.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:53 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:53.585+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:53.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:53 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:53.585+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:53.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:53 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:53.585+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:53.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:53 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:53.585+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:53.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:53 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:53.585+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-27T08:50:26.832952+0000 front 2024-05-27T08:50:26.833042+0000 (oldest deadline 2024-05-27T08:50:52.732774+0000) 2024-05-27T08:50:53.985 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:53 smithi092 podman[74195]: 2024-05-27 08:50:53.598606499 +0000 UTC m=+0.017318927 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:50:53.985 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:53 smithi092 podman[74195]: 2024-05-27 08:50:53.712172222 +0000 UTC m=+0.130884630 container create 293d0096974c2b80d4d8d9ab1ebb98b95f38b72c1c1a3c37b326daf4bc893745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-27T08:50:53.985 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:53 smithi092 podman[74195]: 2024-05-27 08:50:53.92325346 +0000 UTC m=+0.341965861 container init 293d0096974c2b80d4d8d9ab1ebb98b95f38b72c1c1a3c37b326daf4bc893745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:50:53.986 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:53 smithi092 podman[74195]: 2024-05-27 08:50:53.926297114 +0000 UTC m=+0.345009527 container start 293d0096974c2b80d4d8d9ab1ebb98b95f38b72c1c1a3c37b326daf4bc893745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-27T08:50:54.450 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:54 smithi092 podman[74195]: 2024-05-27 08:50:54.014606987 +0000 UTC m=+0.433319401 container attach 293d0096974c2b80d4d8d9ab1ebb98b95f38b72c1c1a3c37b326daf4bc893745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093) 2024-05-27T08:50:54.450 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:54 smithi092 podman[74195]: 2024-05-27 08:50:54.21999882 +0000 UTC m=+0.638711234 container died 293d0096974c2b80d4d8d9ab1ebb98b95f38b72c1c1a3c37b326daf4bc893745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True) 2024-05-27T08:50:54.709 INFO:journalctl@ceph.osd.10.smithi092.stdout:May 27 08:50:54 smithi092 podman[74221]: 2024-05-27 08:50:54.692765176 +0000 UTC m=+0.466989674 container remove 293d0096974c2b80d4d8d9ab1ebb98b95f38b72c1c1a3c37b326daf4bc893745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:50:54.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.10.service' 2024-05-27T08:50:54.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:50:54.759 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-27T08:50:54.759 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-27T08:50:54.760 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.11 2024-05-27T08:50:54.797 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:54 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:54.544+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:54.798 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:54 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:54.544+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:54.798 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:54 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:54.544+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:54.798 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:54 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:54.544+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:54.798 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:54 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:54.544+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:54.798 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:54 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:54.544+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:54.798 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:54 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:54.544+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-27T08:50:26.832952+0000 front 2024-05-27T08:50:26.833042+0000 (oldest deadline 2024-05-27T08:50:52.732774+0000) 2024-05-27T08:50:55.047 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:54 smithi179 systemd[1]: Stopping Ceph osd.11 for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:50:55.047 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.046+0000 7f4533789700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:50:55.048 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.046+0000 7f4533789700 -1 osd.11 99 *** Got signal Terminated *** 2024-05-27T08:50:55.048 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.046+0000 7f4533789700 -1 osd.11 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-27T08:50:55.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.526+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:55.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.526+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:55.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.526+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:55.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.526+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:55.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.526+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:55.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.526+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:55.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:55 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:55.526+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-27T08:50:26.832952+0000 front 2024-05-27T08:50:26.833042+0000 (oldest deadline 2024-05-27T08:50:52.732774+0000) 2024-05-27T08:50:56.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:56 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:56.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:56.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:56 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:56.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:56.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:56 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:56.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:56.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:56 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:56.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:56.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:56 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:56.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:56.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:56 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:56.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:56.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:56 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:56.535+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-27T08:50:26.832952+0000 front 2024-05-27T08:50:26.833042+0000 (oldest deadline 2024-05-27T08:50:52.732774+0000) 2024-05-27T08:50:57.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:57 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:57.545+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:57.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:57 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:57.545+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:57.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:57 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:57.545+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:57.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:57 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:57.545+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:57.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:57 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:57.545+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:57.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:57 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:57.545+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:57.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:57 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:57.545+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-27T08:50:26.832952+0000 front 2024-05-27T08:50:26.833042+0000 (oldest deadline 2024-05-27T08:50:52.732774+0000) 2024-05-27T08:50:58.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:58.502+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:58.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:58.502+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:58.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:58.502+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:58.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:58.502+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:58.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:58.502+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:58.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:58.502+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:58.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:58.502+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-27T08:50:26.832952+0000 front 2024-05-27T08:50:26.833042+0000 (oldest deadline 2024-05-27T08:50:52.732774+0000) 2024-05-27T08:50:58.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:58 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:58.502+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-05-27T08:50:32.733310+0000 front 2024-05-27T08:50:32.733326+0000 (oldest deadline 2024-05-27T08:50:58.033349+0000) 2024-05-27T08:50:59.855 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:59.522+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6806 osd.0 since back 2024-05-27T08:49:33.923631+0000 front 2024-05-27T08:49:33.923600+0000 (oldest deadline 2024-05-27T08:49:58.023591+0000) 2024-05-27T08:50:59.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:59.522+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6804 osd.1 since back 2024-05-27T08:49:40.925005+0000 front 2024-05-27T08:49:40.924811+0000 (oldest deadline 2024-05-27T08:50:05.024874+0000) 2024-05-27T08:50:59.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:59.522+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-05-27T08:49:48.426592+0000 front 2024-05-27T08:49:48.426611+0000 (oldest deadline 2024-05-27T08:50:13.126842+0000) 2024-05-27T08:50:59.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:59.522+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6812 osd.4 since back 2024-05-27T08:50:05.130472+0000 front 2024-05-27T08:50:05.130498+0000 (oldest deadline 2024-05-27T08:50:28.030512+0000) 2024-05-27T08:50:59.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:59.522+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-05-27T08:50:10.931356+0000 front 2024-05-27T08:50:10.931465+0000 (oldest deadline 2024-05-27T08:50:36.831475+0000) 2024-05-27T08:50:59.856 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:59.522+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.77:6822 osd.6 since back 2024-05-27T08:50:16.832115+0000 front 2024-05-27T08:50:16.832039+0000 (oldest deadline 2024-05-27T08:50:42.131880+0000) 2024-05-27T08:50:59.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:59.522+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.92:6820 osd.7 since back 2024-05-27T08:50:26.832952+0000 front 2024-05-27T08:50:26.833042+0000 (oldest deadline 2024-05-27T08:50:52.732774+0000) 2024-05-27T08:50:59.857 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:50:59 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11[61955]: 2024-05-27T08:50:59.522+0000 7f452fd7d700 -1 osd.11 99 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-05-27T08:50:32.733310+0000 front 2024-05-27T08:50:32.733326+0000 (oldest deadline 2024-05-27T08:50:58.033349+0000) 2024-05-27T08:51:00.798 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:00 smithi179 podman[74589]: 2024-05-27 08:51:00.535017305 +0000 UTC m=+5.573270390 container died dcb7b594963971365383dd4eb17c55f930c37c5bf09448667ccb048c4463b2d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-27T08:51:01.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:00 smithi179 podman[74589]: 2024-05-27 08:51:00.879586495 +0000 UTC m=+5.917839568 container cleanup dcb7b594963971365383dd4eb17c55f930c37c5bf09448667ccb048c4463b2d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521) 2024-05-27T08:51:01.089 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:00 smithi179 bash[74589]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11 2024-05-27T08:51:01.355 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:01 smithi179 podman[74605]: 2024-05-27 08:51:01.089218066 +0000 UTC m=+0.549211285 container remove dcb7b594963971365383dd4eb17c55f930c37c5bf09448667ccb048c4463b2d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521) 2024-05-27T08:51:01.750 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:01 smithi179 podman[74692]: 2024-05-27 08:51:01.481851575 +0000 UTC m=+0.103372898 container create fa44bbf51702fa317423591c81e51c89e56c042fca20b5d4820ed8d1fc908403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-27T08:51:01.750 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:01 smithi179 podman[74692]: 2024-05-27 08:51:01.39557748 +0000 UTC m=+0.017098821 image pull 47fc2147e0a3f2c90e07a66b0c5d2d304c3b4fdc90ba885a9434fcc0f291ecc5 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7 2024-05-27T08:51:01.751 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:01 smithi179 podman[74692]: 2024-05-27 08:51:01.675783753 +0000 UTC m=+0.297305097 container init fa44bbf51702fa317423591c81e51c89e56c042fca20b5d4820ed8d1fc908403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:51:01.751 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:01 smithi179 podman[74692]: 2024-05-27 08:51:01.678839915 +0000 UTC m=+0.300361253 container start fa44bbf51702fa317423591c81e51c89e56c042fca20b5d4820ed8d1fc908403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240521, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-27T08:51:01.751 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:01 smithi179 podman[74692]: 2024-05-27 08:51:01.750648734 +0000 UTC m=+0.372170064 container attach fa44bbf51702fa317423591c81e51c89e56c042fca20b5d4820ed8d1fc908403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5) 2024-05-27T08:51:02.105 INFO:journalctl@ceph.osd.11.smithi179.stdout:May 27 08:51:01 smithi179 podman[74692]: 2024-05-27 08:51:01.972802149 +0000 UTC m=+0.594323481 container died fa44bbf51702fa317423591c81e51c89e56c042fca20b5d4820ed8d1fc908403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-27T08:51:02.565 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@osd.11.service' 2024-05-27T08:51:02.643 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:02.644 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-27T08:51:02.644 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-27T08:51:02.644 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.a 2024-05-27T08:51:02.950 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:51:02 smithi077 systemd[1]: Stopping Ceph mds.a for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:03.379 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:51:02 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a[73999]: 2024-05-27T08:51:02.949+0000 7f7084b8a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:03.379 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:51:02 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a[73999]: 2024-05-27T08:51:02.949+0000 7f7084b8a700 -1 mds.a *** got signal Terminated *** 2024-05-27T08:51:07.275 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:51:06 smithi077 podman[104143]: 2024-05-27 08:51:06.955987031 +0000 UTC m=+4.103136774 container died f89bb5c0f0093a6550ed39a8d5fccc4f012e055bbeaf530b7f81d7c0d5f3c537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiesce-a9cb3a5, maintainer=Guillaume Abrioux ) 2024-05-27T08:51:07.628 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:51:07 smithi077 podman[104143]: 2024-05-27 08:51:07.432817111 +0000 UTC m=+4.579966873 container cleanup f89bb5c0f0093a6550ed39a8d5fccc4f012e055bbeaf530b7f81d7c0d5f3c537 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2) 2024-05-27T08:51:07.629 INFO:journalctl@ceph.mds.a.smithi077.stdout:May 27 08:51:07 smithi077 bash[104143]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-a 2024-05-27T08:51:07.865 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.a.service' 2024-05-27T08:51:07.901 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:07.902 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-27T08:51:07.902 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-27T08:51:07.902 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.d 2024-05-27T08:51:08.237 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:51:07 smithi077 systemd[1]: Stopping Ceph mds.d for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:08.628 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:51:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d[72765]: 2024-05-27T08:51:08.236+0000 7f39e3beb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:08.629 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:51:08 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d[72765]: 2024-05-27T08:51:08.236+0000 7f39e3beb700 -1 mds.d *** got signal Terminated *** 2024-05-27T08:51:10.620 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:51:10 smithi077 podman[104276]: 2024-05-27 08:51:10.352209279 +0000 UTC m=+2.222329977 container died 161d1c4d41cdaf583860bcd9ebed343c6e4953127f2187d5b12741a58caf7c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-05-27T08:51:10.878 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:51:10 smithi077 podman[104276]: 2024-05-27 08:51:10.720258519 +0000 UTC m=+2.590379213 container cleanup 161d1c4d41cdaf583860bcd9ebed343c6e4953127f2187d5b12741a58caf7c3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T08:51:10.879 INFO:journalctl@ceph.mds.d.smithi077.stdout:May 27 08:51:10 smithi077 bash[104276]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-d 2024-05-27T08:51:11.147 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.d.service' 2024-05-27T08:51:11.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:11.181 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-27T08:51:11.181 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-27T08:51:11.181 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.g 2024-05-27T08:51:11.628 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:51:11 smithi077 systemd[1]: Stopping Ceph mds.g for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:11.629 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:51:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g[73556]: 2024-05-27T08:51:11.499+0000 7f38e70c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:11.629 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:51:11 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g[73556]: 2024-05-27T08:51:11.499+0000 7f38e70c2700 -1 mds.g *** got signal Terminated *** 2024-05-27T08:51:15.473 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:51:15 smithi077 podman[104396]: 2024-05-27 08:51:15.155424278 +0000 UTC m=+3.754687407 container died 37f65f013fbc28c55398956fbc7e4b040771810c103ee7f68b8bc974978a9745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:51:15.474 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:51:15 smithi077 podman[104396]: 2024-05-27 08:51:15.473979378 +0000 UTC m=+4.073242507 container cleanup 37f65f013fbc28c55398956fbc7e4b040771810c103ee7f68b8bc974978a9745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521) 2024-05-27T08:51:15.878 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:51:15 smithi077 bash[104396]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g 2024-05-27T08:51:15.879 INFO:journalctl@ceph.mds.g.smithi077.stdout:May 27 08:51:15 smithi077 podman[104408]: 2024-05-27 08:51:15.675552658 +0000 UTC m=+0.516399787 container remove 37f65f013fbc28c55398956fbc7e4b040771810c103ee7f68b8bc974978a9745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-27T08:51:15.911 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.g.service' 2024-05-27T08:51:15.943 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:15.944 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-27T08:51:15.944 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-27T08:51:15.944 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.j 2024-05-27T08:51:16.269 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:51:16 smithi077 systemd[1]: Stopping Ceph mds.j for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:16.628 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:51:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-j[72305]: 2024-05-27T08:51:16.269+0000 7f4cc89ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:16.629 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:51:16 smithi077 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-j[72305]: 2024-05-27T08:51:16.269+0000 7f4cc89ec700 -1 mds.j *** got signal Terminated *** 2024-05-27T08:51:18.261 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:51:17 smithi077 podman[104518]: 2024-05-27 08:51:17.99026107 +0000 UTC m=+1.829471629 container died 1d9ce26d82a5b9d4f199d8094ad25ffcd1070cf5faccb56a33f7d24b1102c88c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:51:18.603 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:51:18 smithi077 podman[104518]: 2024-05-27 08:51:18.361438927 +0000 UTC m=+2.200649486 container cleanup 1d9ce26d82a5b9d4f199d8094ad25ffcd1070cf5faccb56a33f7d24b1102c88c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-27T08:51:18.604 INFO:journalctl@ceph.mds.j.smithi077.stdout:May 27 08:51:18 smithi077 bash[104518]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-j 2024-05-27T08:51:18.739 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.j.service' 2024-05-27T08:51:18.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:18.771 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-27T08:51:18.771 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-27T08:51:18.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.b 2024-05-27T08:51:19.174 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:51:18 smithi092 systemd[1]: Stopping Ceph mds.b for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:19.439 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:51:19 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: 2024-05-27T08:51:19.174+0000 7feb76831700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:19.439 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:51:19 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b[66649]: 2024-05-27T08:51:19.174+0000 7feb76831700 -1 mds.b *** got signal Terminated *** 2024-05-27T08:51:23.485 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:51:23 smithi092 podman[74301]: 2024-05-27 08:51:23.165929175 +0000 UTC m=+4.200710338 container died e737fd95726e46f82767804184308de183c0fde7882079cb4d9af85c1e8c4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-05-27T08:51:23.985 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:51:23 smithi092 podman[74301]: 2024-05-27 08:51:23.773815948 +0000 UTC m=+4.808597133 container cleanup e737fd95726e46f82767804184308de183c0fde7882079cb4d9af85c1e8c4f04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-27T08:51:23.985 INFO:journalctl@ceph.mds.b.smithi092.stdout:May 27 08:51:23 smithi092 bash[74301]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-b 2024-05-27T08:51:24.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.b.service' 2024-05-27T08:51:24.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:24.186 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-27T08:51:24.186 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-27T08:51:24.186 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.e 2024-05-27T08:51:24.485 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:51:24 smithi092 systemd[1]: Stopping Ceph mds.e for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:24.985 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:51:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e[66031]: 2024-05-27T08:51:24.518+0000 7fa90734c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:24.985 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:51:24 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e[66031]: 2024-05-27T08:51:24.518+0000 7fa90734c700 -1 mds.e *** got signal Terminated *** 2024-05-27T08:51:27.452 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:51:27 smithi092 podman[74437]: 2024-05-27 08:51:27.151328321 +0000 UTC m=+2.741865887 container died e417bc2498d9314d4cc0842e4d568513b49a09f75daa2880f56e5f0466b73a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-27T08:51:27.735 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:51:27 smithi092 podman[74437]: 2024-05-27 08:51:27.552274455 +0000 UTC m=+3.142812015 container cleanup e417bc2498d9314d4cc0842e4d568513b49a09f75daa2880f56e5f0466b73a32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiesce-a9cb3a5) 2024-05-27T08:51:27.735 INFO:journalctl@ceph.mds.e.smithi092.stdout:May 27 08:51:27 smithi092 bash[74437]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-e 2024-05-27T08:51:27.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.e.service' 2024-05-27T08:51:27.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:27.965 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-27T08:51:27.965 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-27T08:51:27.965 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.h 2024-05-27T08:51:28.235 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:51:28 smithi092 systemd[1]: Stopping Ceph mds.h for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:28.735 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:51:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h[66332]: 2024-05-27T08:51:28.272+0000 7f828ee86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:28.735 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:51:28 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h[66332]: 2024-05-27T08:51:28.272+0000 7f828ee86700 -1 mds.h *** got signal Terminated *** 2024-05-27T08:51:31.449 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:51:31 smithi092 podman[74557]: 2024-05-27 08:51:31.159676561 +0000 UTC m=+2.975257387 container died 2ec7d89d40049ed2a796cb95c5ee11b591ac69a706c2d56e3953498790d17698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.license=GPLv2) 2024-05-27T08:51:31.735 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:51:31 smithi092 podman[74557]: 2024-05-27 08:51:31.547309195 +0000 UTC m=+3.362890022 container cleanup 2ec7d89d40049ed2a796cb95c5ee11b591ac69a706c2d56e3953498790d17698 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux ) 2024-05-27T08:51:31.735 INFO:journalctl@ceph.mds.h.smithi092.stdout:May 27 08:51:31 smithi092 bash[74557]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-h 2024-05-27T08:51:31.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.h.service' 2024-05-27T08:51:31.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:31.962 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-27T08:51:31.963 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-27T08:51:31.963 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.k 2024-05-27T08:51:32.485 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:51:32 smithi092 systemd[1]: Stopping Ceph mds.k for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:32.485 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:51:32 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k[65718]: 2024-05-27T08:51:32.274+0000 7fdafbde1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:32.485 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:51:32 smithi092 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k[65718]: 2024-05-27T08:51:32.274+0000 7fdafbde1700 -1 mds.k *** got signal Terminated *** 2024-05-27T08:51:35.467 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:51:35 smithi092 podman[74680]: 2024-05-27 08:51:35.176106849 +0000 UTC m=+2.999114283 container died d6c6c8fda2d85ed86043eb4ecd6e34218362c41715a11b0bbfccc2f19b2de91c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_CLEAN=True, org.label-schema.build-date=20240521, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-05-27T08:51:35.735 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:51:35 smithi092 podman[74680]: 2024-05-27 08:51:35.550415486 +0000 UTC m=+3.373422919 container cleanup d6c6c8fda2d85ed86043eb4ecd6e34218362c41715a11b0bbfccc2f19b2de91c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-27T08:51:35.735 INFO:journalctl@ceph.mds.k.smithi092.stdout:May 27 08:51:35 smithi092 bash[74680]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-k 2024-05-27T08:51:35.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.k.service' 2024-05-27T08:51:36.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:36.001 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-27T08:51:36.001 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-27T08:51:36.002 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.c 2024-05-27T08:51:36.356 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:51:36 smithi179 systemd[1]: Stopping Ceph mds.c for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:36.855 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:51:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c[67331]: 2024-05-27T08:51:36.361+0000 7f7d95d0f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:36.855 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:51:36 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c[67331]: 2024-05-27T08:51:36.361+0000 7f7d95d0f700 -1 mds.c *** got signal Terminated *** 2024-05-27T08:51:39.605 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:51:39 smithi179 podman[74796]: 2024-05-27 08:51:39.167871786 +0000 UTC m=+2.963523595 container died de3049cdd64342f58f8db1d14f8bdbbfce2c7af9a6e6eb7bea88faf0dea9991b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0) 2024-05-27T08:51:39.995 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:51:39 smithi179 podman[74796]: 2024-05-27 08:51:39.769929579 +0000 UTC m=+3.565581384 container cleanup de3049cdd64342f58f8db1d14f8bdbbfce2c7af9a6e6eb7bea88faf0dea9991b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240521, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, ceph=True) 2024-05-27T08:51:39.996 INFO:journalctl@ceph.mds.c.smithi179.stdout:May 27 08:51:39 smithi179 bash[74796]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-c 2024-05-27T08:51:40.129 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.c.service' 2024-05-27T08:51:40.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:40.163 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-27T08:51:40.164 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-27T08:51:40.164 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.f 2024-05-27T08:51:40.355 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:40 smithi179 systemd[1]: Stopping Ceph mds.f for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:40.855 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:40 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f[67032]: 2024-05-27T08:51:40.473+0000 7f4317347700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:40.855 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:40 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f[67032]: 2024-05-27T08:51:40.473+0000 7f4317347700 -1 mds.f *** got signal Terminated *** 2024-05-27T08:51:43.490 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:43 smithi179 podman[74928]: 2024-05-27 08:51:43.157612934 +0000 UTC m=+2.777948416 container died 0b34f7b54725ae2af132814851ebe198a85705ce7f84d68edbdbde0026a5c518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-27T08:51:43.774 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:43 smithi179 podman[74928]: 2024-05-27 08:51:43.590130173 +0000 UTC m=+3.210465664 container cleanup 0b34f7b54725ae2af132814851ebe198a85705ce7f84d68edbdbde0026a5c518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-27T08:51:43.774 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:43 smithi179 bash[74928]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f 2024-05-27T08:51:44.018 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.f.service' 2024-05-27T08:51:44.038 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:43 smithi179 podman[74942]: 2024-05-27 08:51:43.774562448 +0000 UTC m=+0.609782337 container remove 0b34f7b54725ae2af132814851ebe198a85705ce7f84d68edbdbde0026a5c518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-27T08:51:44.039 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:44 smithi179 systemd[1]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.f.service: Deactivated successfully. 2024-05-27T08:51:44.039 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:44 smithi179 systemd[1]: Stopped Ceph mds.f for c4eeaaa6-1c02-11ef-bc9b-c7b262605968. 2024-05-27T08:51:44.039 INFO:journalctl@ceph.mds.f.smithi179.stdout:May 27 08:51:44 smithi179 systemd[1]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.f.service: Consumed 33.946s CPU time. 2024-05-27T08:51:44.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:44.050 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-27T08:51:44.051 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-27T08:51:44.051 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.i 2024-05-27T08:51:44.356 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:51:44 smithi179 systemd[1]: Stopping Ceph mds.i for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:44.855 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:51:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i[67645]: 2024-05-27T08:51:44.376+0000 7fbff9406700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:44.900 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:51:44 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i[67645]: 2024-05-27T08:51:44.376+0000 7fbff9406700 -1 mds.i *** got signal Terminated *** 2024-05-27T08:51:47.502 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:51:47 smithi179 podman[75048]: 2024-05-27 08:51:47.162564394 +0000 UTC m=+2.884812631 container died b660e7f6b09bead3af680a0df2be69071bdea27ce9b911fabdd2a0887d482f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, org.label-schema.vendor=CentOS, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240521, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-27T08:51:47.855 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:51:47 smithi179 podman[75048]: 2024-05-27 08:51:47.618783777 +0000 UTC m=+3.341032011 container cleanup b660e7f6b09bead3af680a0df2be69071bdea27ce9b911fabdd2a0887d482f97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiesce-a9cb3a5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240521, ceph=True) 2024-05-27T08:51:47.856 INFO:journalctl@ceph.mds.i.smithi179.stdout:May 27 08:51:47 smithi179 bash[75048]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-i 2024-05-27T08:51:48.046 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.i.service' 2024-05-27T08:51:48.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:48.078 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-27T08:51:48.078 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-27T08:51:48.079 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.l 2024-05-27T08:51:48.397 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:51:48 smithi179 systemd[1]: Stopping Ceph mds.l for c4eeaaa6-1c02-11ef-bc9b-c7b262605968... 2024-05-27T08:51:48.857 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:51:48 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l[66729]: 2024-05-27T08:51:48.396+0000 7fdd24d57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-27T08:51:48.857 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:51:48 smithi179 ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l[66729]: 2024-05-27T08:51:48.396+0000 7fdd24d57700 -1 mds.l *** got signal Terminated *** 2024-05-27T08:51:51.439 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:51:51 smithi179 podman[75169]: 2024-05-27 08:51:51.172635874 +0000 UTC m=+2.877597644 container died 4746f09de9fab0604147135662803d976a1ed124818409a9f28d990d18294b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240521, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiesce-a9cb3a5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-27T08:51:51.797 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:51:51 smithi179 podman[75169]: 2024-05-27 08:51:51.53860092 +0000 UTC m=+3.243562677 container cleanup 4746f09de9fab0604147135662803d976a1ed124818409a9f28d990d18294b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7500038023c55c6f395b6d2464296e75584a921fdb356c915ce07a53884817b7, name=ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiesce-a9cb3a5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e5230d3b004eecd2f42a9b578ebb9642654e8093, org.label-schema.build-date=20240521, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-27T08:51:51.797 INFO:journalctl@ceph.mds.l.smithi179.stdout:May 27 08:51:51 smithi179 bash[75169]: ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968-mds-l 2024-05-27T08:51:51.950 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c4eeaaa6-1c02-11ef-bc9b-c7b262605968@mds.l.service' 2024-05-27T08:51:51.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-27T08:51:51.981 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-27T08:51:51.982 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:51:52.172 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:51:54.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:51:54.784 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:51:57.192 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 --force --keep-logs 2024-05-27T08:51:57.381 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:51:59.715 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:51:59.747 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:51:59.779 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-27T08:51:59.805 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-27T08:51:59.807 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi077/crash 2024-05-27T08:51:59.808 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:51:59.855 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:51:59.855 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:51:59.858 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi092/crash 2024-05-27T08:51:59.858 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:51:59.903 INFO:teuthology.orchestra.run.smithi092.stderr:tar: /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:51:59.904 INFO:teuthology.orchestra.run.smithi092.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:51:59.905 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi179/crash 2024-05-27T08:51:59.906 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash -- . 2024-05-27T08:51:59.943 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-05-27T08:51:59.943 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-05-27T08:51:59.944 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-27T08:51:59.944 DEBUG:teuthology.orchestra.run.smithi077:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-27T08:51:59.987 INFO:tasks.cephadm:Compressing logs... 2024-05-27T08:51:59.987 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:52:00.030 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:52:00.033 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-27T08:52:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:52:00.055 INFO:teuthology.orchestra.run.smithi077.stderr:: No such file or directory 2024-05-27T08:52:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:52:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-05-27T08:52:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-05-27T08:52:00.056 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-05-27T08:52:00.066 INFO:teuthology.orchestra.run.smithi092.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:52:00.067 INFO:teuthology.orchestra.run.smithi092.stderr:: No such file or directory 2024-05-27T08:52:00.067 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:52:00.067 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-05-27T08:52:00.067 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-05-27T08:52:00.067 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-05-27T08:52:00.067 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-05-27T08:52:00.067 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.b.log: /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-05-27T08:52:00.068 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-05-27T08:52:00.068 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-05-27T08:52:00.068 INFO:teuthology.orchestra.run.smithi092.stderr: 86.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T08:52:00.068 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.h.log: 90.5% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:52:00.068 INFO:teuthology.orchestra.run.smithi092.stderr: 91.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-05-27T08:52:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:52:00.074 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-05-27T08:52:00.075 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:52:00.075 INFO:teuthology.orchestra.run.smithi092.stderr: 83.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:52:00.077 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-05-27T08:52:00.077 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.j.log: /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-05-27T08:52:00.077 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-05-27T08:52:00.077 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-05-27T08:52:00.078 INFO:teuthology.orchestra.run.smithi077.stderr: 86.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log.gz/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.a.log: 2024-05-27T08:52:00.078 INFO:teuthology.orchestra.run.smithi077.stderr: 90.3% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:52:00.078 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:52:00.078 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-05-27T08:52:00.078 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log: /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.6.log: 84.2% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:52:00.079 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:52:00.079 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.a.log: 94.7% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-05-27T08:52:00.079 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-05-27T08:52:00.079 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-05-27T08:52:00.079 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.d.log: 91.1% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:52:00.079 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:52:00.079 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log 2024-05-27T08:52:00.080 INFO:teuthology.orchestra.run.smithi179.stderr:: No such file or directory 2024-05-27T08:52:00.080 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-05-27T08:52:00.080 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-05-27T08:52:00.080 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log: /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log 2024-05-27T08:52:00.080 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-05-27T08:52:00.080 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-05-27T08:52:00.081 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-05-27T08:52:00.081 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-05-27T08:52:00.081 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-05-27T08:52:00.081 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.z.log: 86.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.log.gz 2024-05-27T08:52:00.081 INFO:teuthology.orchestra.run.smithi179.stderr: 90.5% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-05-27T08:52:00.081 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-05-27T08:52:00.081 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-05-27T08:52:00.082 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-05-27T08:52:00.082 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-05-27T08:52:00.082 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-05-27T08:52:00.082 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log 2024-05-27T08:52:00.083 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-05-27T08:52:00.089 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-05-27T08:52:00.091 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-05-27T08:52:00.095 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-05-27T08:52:00.097 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:52:00.098 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:52:00.098 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-27T08:52:00.108 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/cephadm.log: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:52:00.112 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:52:00.116 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-05-27T08:52:00.123 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-27T08:52:00.429 INFO:teuthology.orchestra.run.smithi077.stderr: 90.1% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-05-27T08:52:00.445 INFO:teuthology.orchestra.run.smithi077.stderr: 94.5% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-05-27T08:52:00.483 INFO:teuthology.orchestra.run.smithi077.stderr: 94.7% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-05-27T08:52:00.509 INFO:teuthology.orchestra.run.smithi179.stderr: 92.0% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-05-27T08:52:00.645 INFO:teuthology.orchestra.run.smithi092.stderr: 91.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-05-27T08:52:01.114 INFO:teuthology.orchestra.run.smithi077.stderr: 90.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-05-27T08:52:01.198 INFO:teuthology.orchestra.run.smithi179.stderr: 95.7% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-05-27T08:52:01.403 INFO:teuthology.orchestra.run.smithi092.stderr: 95.4% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-05-27T08:52:01.985 INFO:teuthology.orchestra.run.smithi092.stderr: 95.8% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-05-27T08:52:02.029 INFO:teuthology.orchestra.run.smithi077.stderr: 93.8% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-05-27T08:52:02.225 INFO:teuthology.orchestra.run.smithi077.stderr: 93.7% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-05-27T08:52:02.443 INFO:teuthology.orchestra.run.smithi077.stderr: 93.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-05-27T08:52:02.487 INFO:teuthology.orchestra.run.smithi179.stderr: 93.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-05-27T08:52:02.565 INFO:teuthology.orchestra.run.smithi092.stderr: 93.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-05-27T08:52:02.610 INFO:teuthology.orchestra.run.smithi077.stderr: 93.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-05-27T08:52:02.612 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-05-27T08:52:02.612 INFO:teuthology.orchestra.run.smithi077.stderr:real 0m2.569s 2024-05-27T08:52:02.612 INFO:teuthology.orchestra.run.smithi077.stderr:user 0m10.911s 2024-05-27T08:52:02.613 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m0.426s 2024-05-27T08:52:02.861 INFO:teuthology.orchestra.run.smithi092.stderr: 94.0% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-05-27T08:52:03.162 INFO:teuthology.orchestra.run.smithi092.stderr: 93.9% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-05-27T08:52:03.837 INFO:teuthology.orchestra.run.smithi179.stderr: 94.0% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-05-27T08:52:03.877 INFO:teuthology.orchestra.run.smithi179.stderr: 94.3% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-05-27T08:52:04.442 INFO:teuthology.orchestra.run.smithi092.stderr: 94.2% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-05-27T08:52:04.527 INFO:teuthology.orchestra.run.smithi179.stderr: 94.2% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-05-27T08:52:06.161 INFO:teuthology.orchestra.run.smithi092.stderr: 94.4% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-05-27T08:52:08.213 INFO:teuthology.orchestra.run.smithi179.stderr: 94.2% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-05-27T08:52:15.703 INFO:teuthology.orchestra.run.smithi179.stderr: 94.0% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-05-27T08:52:17.635 INFO:teuthology.orchestra.run.smithi179.stderr: 94.1% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-05-27T08:52:17.636 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-05-27T08:52:17.637 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m17.590s 2024-05-27T08:52:17.637 INFO:teuthology.orchestra.run.smithi179.stderr:user 0m55.113s 2024-05-27T08:52:17.637 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m1.805s 2024-05-27T08:52:32.737 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/c4eeaaa6-1c02-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-05-27T08:52:32.738 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-05-27T08:52:32.738 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m32.692s 2024-05-27T08:52:32.739 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m52.980s 2024-05-27T08:52:32.739 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m1.761s 2024-05-27T08:52:32.739 INFO:tasks.cephadm:Archiving logs... 2024-05-27T08:52:32.739 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi077/log 2024-05-27T08:52:32.741 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:52:34.145 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi092/log 2024-05-27T08:52:34.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:52:40.869 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi179/log 2024-05-27T08:52:40.870 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-27T08:52:47.495 INFO:tasks.cephadm:Removing cluster... 2024-05-27T08:52:47.495 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 --force 2024-05-27T08:52:47.691 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:52:48.015 DEBUG:teuthology.orchestra.run.smithi092:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 --force 2024-05-27T08:52:48.204 INFO:teuthology.orchestra.run.smithi092.stdout:Deleting cluster with fsid: c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:52:48.527 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c4eeaaa6-1c02-11ef-bc9b-c7b262605968 --force 2024-05-27T08:52:48.768 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: c4eeaaa6-1c02-11ef-bc9b-c7b262605968 2024-05-27T08:52:49.089 INFO:tasks.cephadm:Removing cephadm ... 2024-05-27T08:52:49.089 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:52:49.110 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:52:49.127 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-27T08:52:49.143 INFO:tasks.cephadm:Teardown complete 2024-05-27T08:52:49.144 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-27T08:52:49.157 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-27T08:52:49.157 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:52:49.160 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:52:49.169 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-27T08:52:49.273 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-27T08:52:49.284 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-27T08:52:49.285 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:52:49.287 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:52:49.290 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-27T08:52:49.302 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:52:49.302 INFO:teuthology.orchestra.run.smithi092.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:52:49.303 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-05-27T08:52:49.306 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:52:49.306 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-05-27T08:52:49.306 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 39 +683us[ +683us] +/- 57ms 2024-05-27T08:52:49.306 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv02.front.sepia.ceph.com 3 8 377 104 -777us[ -769us] +/- 68ms 2024-05-27T08:52:49.306 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 103 +383us[ +383us] +/- 70ms 2024-05-27T08:52:49.307 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:52:49.309 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:52:49.309 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-05-27T08:52:49.309 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 39 +562us[ +563us] +/- 57ms 2024-05-27T08:52:49.309 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -920us[ -919us] +/- 69ms 2024-05-27T08:52:49.309 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +261us[ +262us] +/- 71ms 2024-05-27T08:52:49.309 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:52:49.310 INFO:teuthology.orchestra.run.smithi092.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-27T08:52:49.310 INFO:teuthology.orchestra.run.smithi092.stdout:=============================================================================== 2024-05-27T08:52:49.310 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 38 +570us[ +570us] +/- 57ms 2024-05-27T08:52:49.310 INFO:teuthology.orchestra.run.smithi092.stdout:^* hv02.front.sepia.ceph.com 3 7 377 101 -877us[ -840us] +/- 68ms 2024-05-27T08:52:49.310 INFO:teuthology.orchestra.run.smithi092.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +278us[ +278us] +/- 71ms 2024-05-27T08:52:49.310 INFO:teuthology.orchestra.run.smithi092.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-27T08:52:49.313 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-27T08:52:49.324 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-27T08:52:49.325 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-27T08:52:49.336 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:52:49.430 DEBUG:teuthology.orchestra.run.smithi092:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:52:49.496 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-27T08:52:49.561 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:52:49.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:52:49.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:52:49.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:52:49.618 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-27T08:52:49.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:52:49.645 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-27T08:52:49.656 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-27T08:52:49.668 INFO:teuthology.task.internal:Duration was 2018.008662 seconds 2024-05-27T08:52:49.668 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-27T08:52:49.679 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-27T08:52:49.679 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:52:49.682 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:52:49.684 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-27T08:52:49.718 INFO:teuthology.orchestra.run.smithi092.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:52:49.719 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:52:49.722 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-27T08:52:50.026 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-27T08:52:50.026 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-05-27T08:52:50.026 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:52:50.047 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-05-27T08:52:50.047 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:52:50.086 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-05-27T08:52:50.087 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-27T08:52:50.111 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-27T08:52:50.112 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:52:50.114 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:52:50.130 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-27T08:52:50.177 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-27T08:52:50.177 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:52:50.182 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:52:50.195 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-27T08:52:50.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-27T08:52:50.909 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-27T08:52:50.909 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:52:50.935 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:52:50.959 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-27T08:52:50.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-27T08:52:50.995 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:52:50.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:52:51.002 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:52:51.018 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-05-27T08:52:51.022 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-05-27T08:52:51.046 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-05-27T08:52:51.084 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:52:51.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:52:51.099 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:52:51.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:52:51.114 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-27T08:52:51.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-27T08:52:51.138 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-27T08:52:51.150 INFO:teuthology.task.internal:Transferring archived files... 2024-05-27T08:52:51.154 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi077 2024-05-27T08:52:51.155 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:52:51.210 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi092 2024-05-27T08:52:51.210 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:52:51.245 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-05-27_07:25:29-fs-wip-lusov-quiesce-distro-default-smithi/7727776/remote/smithi179 2024-05-27T08:52:51.246 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-27T08:52:51.282 INFO:teuthology.task.internal:Removing archive directory... 2024-05-27T08:52:51.282 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:52:51.285 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:52:51.287 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-27T08:52:51.331 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-27T08:52:51.343 INFO:teuthology.task.internal:Not uploading archives. 2024-05-27T08:52:51.343 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-27T08:52:51.424 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-27T08:52:51.425 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:52:51.427 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:52:51.429 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-27T08:52:51.440 INFO:teuthology.orchestra.run.smithi077.stdout: 265308 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:52 /home/ubuntu/cephtest 2024-05-27T08:52:51.442 INFO:teuthology.orchestra.run.smithi092.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:52 /home/ubuntu/cephtest 2024-05-27T08:52:51.443 INFO:teuthology.orchestra.run.smithi179.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 27 08:52 /home/ubuntu/cephtest 2024-05-27T08:52:51.445 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-27T08:52:51.457 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-27T08:52:51.498 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2018.0086619853973 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi077 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a9cb3a581a309a99b72997ae5ddb88084f5484c9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=adc9ceb56d364867a7c3b5c4b5ae07a8 status: fail success: false 2024-05-27T08:52:51.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-27T08:52:51.591 INFO:teuthology.run:FAIL